Studierfenster

{{Use mdy dates|date=January 2012}}

{{Infobox software

| name = StudierFenster

| logo = Studierfenster.png

| logo caption = The Studierfenster Logo

| screenshot =

| caption =

| developer = [https://www.ikim.uk-essen.de/ University Hospital Essen], [https://www.tugraz.at/home/ Graz University of Technology], [https://www.medunigraz.at/ Medical University of Graz]

| released = {{Start date and age|2018}}

| language = English

| operating system = Cross-platform
(Windows, Mac OS X, Linux)

| programming language = C, C++, Python, JavaScript, HTML

| genre = Image processing, scientific visualization, medical imaging, volume rendering, Interactive visualization

| license = GPL, CC BY-SA

| website = {{URL|http://studierfenster.icg.tugraz.at/}}

}}

File:Studierfenster Brain Tumor Segmentation.png

File:Aortic Dissection Inpainting.png

Studierfenster or StudierFenster (SF){{cite web|url=http://www.studierfenster.at|title=Studierfenster|access-date=2020-04-23}}{{Cite journal|last1=Egger|first1=Jan|last2=Wild|first2=Daniel|last3=Weber|first3=Maximilian|last4=Ramirez Bedoya|first4=Christopher|last5=Karner|first5=Florian|last6=Prutsch|first6=Alexander|last7=Schmied|first7=Michael|last8=Dionysio|first8=Christina|last9=Krobath|first9=Dominik|last10=Jin|first10=Yuan|last11=Gsaxner|first11=Christina|last12=Li|first12=Jianning|last13=Pepe|first13=Antonio|date=2022|title=Studierfenster: an Open Science Cloud-Based Medical Imaging Analysis Platform|journal=Journal of Digital Imaging|volume=35 |issue=2 |language=en|location=Cham|publisher=Springer International Publishing|pages=340–355|doi=10.1007/s10278-021-00574-8|pmid=35064372|pmc=8782222}}{{Cite book|last=Weber|first=Maximilian|chapter=A Client/Server based Online Environment for the Calculation of Medical Segmentation Scores |s2cid=199373900|title=2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)|date=2019-10-17|volume=2019|pages=3463–3467|doi=10.1109/EMBC.2019.8856481|pmid=31946624|isbn=978-1-5386-1311-5|language=en-US}} is a free, non-commercial open science client/server-based medical imaging processing online framework. It offers capabilities, like viewing medical data (computed tomography (CT), magnetic resonance imaging (MRI), etc.) in two- and three-dimensional space directly in the standard web browsers, like Google Chrome, Mozilla Firefox, Safari, and Microsoft Edge. Other functionalities are the calculation of medical metrics (dice score{{cite journal |last=Dice |first=Lee R. |title=Measures of the Amount of Ecologic Association Between Species |jstor=1932409 |journal=Ecology |volume=26 |issue=3 |year=1945 |pages=297–302 |doi=10.2307/1932409 |bibcode=1945Ecol...26..297D |s2cid=53335638 }} and Hausdorff distance{{cite book |authorlink=R. Tyrrell Rockafellar |first1=R. Tyrrell |last1=Rockafellar |authorlink2=Roger J-B Wets |first2=Roger J-B |last2=Wets |title=Variational Analysis |publisher=Springer-Verlag |year=2005 |isbn=3-540-62772-3 |page=117 }}), manual slice-by-slice outlining of structures in medical images (segmentationLinda G. Shapiro and George C. Stockman (2001): “Computer Vision”, pp 279–325, New Jersey, Prentice-Hall, {{ISBN|0-13-030796-3}}Barghout, Lauren, and Lawrence W. Lee. "Perceptual information processing system." Paravue Inc. U.S. Patent Application 10/618,543, filed July 11, 2003.), manual placing of (anatomical) landmarks in medical image data, viewing medical data in virtual reality, a facial reconstruction and registration of medical data for augmented reality,{{Cite book|last1=Gsaxner|first1=Christina|last2=Pepe|first2=Antonio|last3=Wallner|first3=Jürgen|last4=Schmalstieg|first4=Dieter|last5=Egger|first5=Jan|title=Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 |chapter=Markerless Image-to-Face Registration for Untethered Augmented Reality in Head and Neck Surgery |date=2019|editor-last=Shen|editor-first=Dinggang|editor2-last=Liu|editor2-first=Tianming|editor3-last=Peters|editor3-first=Terry M.|editor4-last=Staib|editor4-first=Lawrence H.|editor5-last=Essert|editor5-first=Caroline|editor6-last=Zhou|editor6-first=Sean|editor7-last=Yap|editor7-first=Pew-Thian|editor8-last=Khan|editor8-first=Ali|series=Lecture Notes in Computer Science|volume=11768|language=en|location=Cham|publisher=Springer International Publishing|pages=236–244|doi=10.1007/978-3-030-32254-0_27|isbn=978-3-030-32254-0|s2cid=202620139}} one click showcases for COVID-19 and veterinary scans, and a Radiomics module.

Other features of Studierfenster are the automatic cranial implant design with a neural network,{{Cite thesis|url=https://graz.pure.elsevier.com/en/publications/deep-learning-for-cranial-defect-reconstruction | title= Deep Learning for Cranial Defect Reconstruction|last=Li|first=Jianning|website=Master Thesis, Institute of Computer Graphics and Vision, Graz University of Technology, Austria, pp. 1-68, January 2020.| date= January 2020| type= Master's Thesis}}{{Cite book|arxiv=2006.00980|last1=Li|first1=Jianning|last2=Pepe|first2=Antonio|last3=Gsaxner|first3=Christina|last4=Egger|first4=Jan|title=Medical Imaging 2021: Image-Guided Procedures, Robotic Interventions, and Modeling |chapter=An online platform for automatic skull defect restoration and cranial implant design |editor-first1=Cristian A. |editor-first2=Jeffrey H. |editor-last1=Linte |editor-last2=Siewerdsen |journal=Medical Imaging 2021: Image-Guided Procedures |year=2021|volume=11598 |page=59 |doi=10.1117/12.2580719 |bibcode=2021SPIE11598E..1QL |isbn=9781510640252 |s2cid=219177447 }} the inpainting of aortic dissections{{Cite journal |doi = 10.1016/j.media.2020.101773|title = Detection, Segmentation, Simulation and Visualization of Aortic Dissections: A Review|year = 2020|last1 = Pepe|first1 = Antonio|last2 = Li|first2 = Jianning|last3 = Rolf-Pissarczyk|first3 = Malte|last4 = Gsaxner|first4 = Christina|last5 = Chen|first5 = Xiaojun|last6 = Holzapfel|first6 = Gerhard A.|last7 = Egger|first7 = Jan|journal = Medical Image Analysis|volume = 65|page = 101773|pmid = 32738647|s2cid = 220924735}} with a generative adversarial network,{{Cite web|url=https://cescg.org/wp-content/uploads/2020/03/Prutsch-Design-and-Development-of-a-Web-based-Tool-for-Inpainting-of-Dissected-Aortae-in-Angiography-Images.pdf|title=Design and Development of a Web-based Tool for Inpainting ofDissected Aortae in Angiography Images|last=Prutsch|first=Alexander|access-date=2020-04-25}}{{cite conference|title=Generative Adversarial Networks|first1=Ian |last1=Goodfellow |first2=Jean |last2=Pouget-Abadie |first3=Mehdi |last3=Mirza |first4=Bing |last4=Xu |first5=David |last5=Warde-Farley |first6=Sherjil |last6=Ozair |first7=Aaron |last7=Courville |first8=Yoshua |last8=Bengio |conference= Proceedings of the International Conference on Neural Information Processing Systems (NIPS 2014) |pages= 2672–2680 |url= https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf |year=2014}} an automatic aortic landmark detection with deep learning{{cite journal|last=Schmidhuber|first=J.|s2cid=11715509|year=2015|title=Deep Learning in Neural Networks: An Overview|journal=Neural Networks|volume=61|pages=85–117|arxiv=1404.7828|doi=10.1016/j.neunet.2014.09.003|pmid=25462637}} in computed tomography angiography scans, and a GrowCut algorithm implementation for image segmentation.

Studierfenster is currently hosted on a server at the Graz University of Technology{{cite web|url=https://www.tugraz.at/|title=Graz University of Technology|access-date=2020-04-28}} in Austria, and expanded jointly with the Institute for Artificial Intelligence in Medicine (IKIM){{cite web|url=https://www.ikim.uk-essen.de/|title=Artificial Intelligence in Medicine|access-date=2022-02-09}} in Essen, Germany.

History

Studierfenster was initiated within two bachelor theses during the summer bachelor program of the Institute of Computer Graphics and Vision (ICG) at Graz University of Technology, Austria, in cooperation with the Medical University of Graz, Austria, in 2018/2019.{{Cite thesis|title=A Client/Server based Online Environment for the calculation of Segmentation Scores|last=Weber|first=Maximilian|date=2018-12-13|type=Bachelor Thesis |publisher=Institute of Computer Graphics and Vision, Graz University of Technology |location=Austria |pages=1–40}}{{Cite arXiv|eprint=1904.08610|last1=Wild|first1=Daniel|last2=Weber|first2=Maximilian|last3=Egger|first3=Jan|title=Client/Server Based Online Environment for Manual Segmentation of Medical Images|year=2019|class=cs.CV}}

The name Studierfenster (or StudierFenster) is German and can be translated to 'StudyWindow', whereby window refers here to a browser window. The word Studierfenster is an adaption from the word {{Lang|de|Studierstube}} ('study room'), which was an augmented reality project at the Vienna University of Technology in Austria.{{cite web|url=https://www.cg.tuwien.ac.at/research/vr/studierstube/jvrs-paper.pdf|title=Studierstube|access-date=2020-04-26}}{{Cite journal|last1=Szalavári|first1=Zsolt|last2=Schmalstieg|first2=Dieter|last3=Fuhrmann|first3=Anton|last4=Gervautz|first4=Michael|title=?Studierstube?: An environment for collaboration in augmented reality |s2cid=1122975|date=1998|journal=Virtual Reality |series=Lecture Notes in Computer Science|volume=3|language=en|publisher=Springer International Publishing|pages=37–48|doi=10.1007/BF01409796}}

Architecture

File:Studierfenster Architecture.svg

Studierfenster is set up as a distributed application via a client–server model. The client side (front-end) consists of HTML and JavaScript with WebGL to enable 2D and 3D visualization, rendered on the client.

The server side (back-end) handles client requests via C, C++ and Python.{{cite web|url=https://www.python.org/|title=Python|access-date=2020-04-29}} It interfaces to common open source libraries and software tools like the Insight Toolkit,{{cite web|url=https://itk.org/|title=The Insight Toolkit|access-date=2020-04-27}} the Visualization Toolkit (VTK),{{cite web|url=https://vtk.org/|title=VTK - The Visualization Toolkit|access-date=2020-04-27}} the X Toolkit (XTK){{cite web|url=https://github.com/xtk/X|title=The X Toolkit: WebGL™ for Scientific Visualization|website=GitHub|date=25 April 2020|access-date=2020-04-27}} and Slice:Drop.{{cite web|url=https://slicedrop.com/|title=Slice:Drop|access-date=2020-04-27}} The server communication is handled by AJAX requests{{Cite web|url=https://developer.mozilla.org/en-US/docs/Web/Guide/AJAX|title=Ajax - Web developer guides|website=MDN Web Docs|access-date=2018-02-27|archive-url=https://web.archive.org/web/20180228161512/https://developer.mozilla.org/en-US/docs/Web/Guide/AJAX|archive-date=28 February 2018|url-status=live}} were needed.

Studierfenster employs a Flask server.

Features

= Dicom browser =

File:Studierfenster DICOM Browser.png

This allows client-side parsing a local folder with DICOM (Digital Imaging and Communications in Medicine){{cite web|url=http://dicom.nema.org/medical/dicom/current/output/chtml/part01/chapter_1.html#sect_1.1|title=1 Scope and Field of Application|website=dicom.nema.org}}[http://medical.nema.org/dicom/geninfo/Brochure.pdf DICOM brochure], nema.org. files. Afterwards, the whole folder can be converted to compressed {{Not a typo|.nrrd}} (nearly raw raster data) files and downloaded as a single {{Not a typo|.zip}} file.

Nrrd is a library and file format for the representation and processing of n-dimensional raster data. It is intended to support scientific visualization and (medical) image processing applications.{{cite book | title=Tensors in Image Processing and Computer Vision | last1= Aja-Fernández | first1= Santiago | last2= de Luis Garcia| first2= Rodrigo | last3= Tao |first3= Dacheng|last4= Li|first4=Xuelong | isbn=9781848822993 | series=Advances in Computer Vision and Pattern Recognition | year=2009 | publisher=Springer Science & Business Media }} With the "Dicom Browser" of Studierfenster, it is possible to select specific Studies or Series, and only convert these.

= File converter =

The file converter converts a medical volume file (e.g. a non-compressed {{Not a typo|.nrrd}} file) to a compressed/binary {{Not a typo|.nrrd}} file. After the conversion, the compressed {{Not a typo|.nrrd}} file can be downloaded and used with the "Medical 3D Viewer" for 2D and 3D visualization, and further image processing.

= Metrics module =

File:Studierfenster Metrics Module.png

This can calculate the Dice similarity coefficient and Hausdorff distance between two segmentation masks (in {{Not a typo|.nrrd}} format) in a standard web browser.

The resulting table has seven columns: the file names for both files used in the calculation, the calculated Dice similarity coefficient, the calculated Hausdorff distance, the calculated directed HD for both directions, and the information if image spacing was used in the calculation. The table can be sorted, is searchable, and can be exported as a simple copy, an Excel spreadsheet, a comma-separated values file or as a portable document format.

The Metrics Module has been used to compare manual anatomical segmentations of brain tumors{{Cite journal|last1=Bhandari|first1=Abhishta|last2=Koppen|first2=Jarrad|last3=Agzarian|first3=Marc|date=2020|title=Convolutional neural networks for brain tumour segmentation|journal=Insights into Imaging|series=Lecture Notes in Computer Science|volume=11:77|issue=1|page=77|language=en|publisher=Springer Open|doi=10.1186/s13244-020-00869-4|pmid=32514649|pmc=7280397 |doi-access=free }}

= VR viewer =

File:Studierfenster Virtual Reality.png

The VR Viewer (or Medical VR Viewer) enables viewing (medical) data in Virtual Reality (VR) with devices like the Google Cardboard or the HTC Vive (via the WebVR App).{{Cite journal|title=HTC Vive MeVisLab integration via OpenVR for medical applications|last=Egger|first=Jan|journal=PLOS ONE|date=2017-03-12|volume=12|issue=3|pages=e0173972|doi=10.1371/journal.pone.0173972|pmid=28323840|pmc=5360258|arxiv=1703.07575|bibcode=2017PLoSO..1273972E|language=en-US|doi-access=free}} For viewing the data in VR, it needs to be converted to the VTI (.vti) format, which can be done with open-source, multi-platform data analysis and visualization application ParaView{{cite web|url=https://www.paraview.org|title=ParaView|access-date=2020-05-24}}

Critics

Studierfenster is not a certified medical product; it can only be used for educational, research, and informational purposes.

References

{{Reflist}}