Int J CARS DOI 10.1007/s11548-016-1412-5

ABSTRACTS

CARS 2016—Computer Assisted Radiology and Surgery Proceedings of the 30th International Congress and Exhibition Heidelberg, Germany, June 21–25, 2016  CARS 2016

Preface: CARS Fundamentals

Vision 2. IFCARS aims to provide a platform (Journal and Congress) to communicate and integrate the R&D activities of various medical/technical disciplines in their endeavour to improve patient care in medical diagnostics and therapy. The universe of discourse for which this platform is being evolved is defined in the CARS ontolology/classification.

Spirit

Background 1. Founded in 1983, Computer Assisted Radiology and Surgery (CARS) has pursued a mission to hold a leading role in medical and imaging technology That mission has now a proven track record for more than 30 years. This success has been realized by focusing on research and development on novel algorithms and systems and their applications in radiology and surgery. (a)

The CARS Congress is a yearly event where a renowned international community of scientists, engineers and physicians come together with a common purpose to present and discuss the key innovations that shape modern medicine on a worldwide basis. (b) The International Journal for Computer Assisted Radiology and Surgery (IJCARS) is a peer-reviewed journal that provides a platform for closing the gap between medical and technical disciplines, and promotes interdisciplinary research and development activities focusing on making impact for the global medical environment. The development of CARS is decisively influenced by medical and technical innovation as well as by interdisciplinary and international cooperation. The activities of CARS, including IJCARS and the Congress activities, are in accordance with the founding charter of the International Foundation of CARS (IFCARS) as authorized by the Federal Republic of Germany.

3. All parties involved in building up the CARS platform are expected to work towards the goal of closely networked and or interlinked congress sessions and workshops (as opposed to a federation of independent congress parts). CARS will ascertain high quality presentations in the Congress and publications in the Journal

Skills 4. The competences for realizing the CARS vision and spirit come mainly from members of the IJCARS Deputy Editors and Editorial Board as well as from the IFCARS Congress Organising and Program Committees. These competences are augmented by a multitude of authors worldwide, who actively support the vision and spirit of CARS.

Resources 5. The coordination and back-up for the support of the vision and spirit of CARS and associated processes and action plans comes from IFCARS.

Process/Action Plan 6. The handling of submissions for CARS (long abstracts as well as full papers) generally proceeds along a given framework: (a)

Original IT-based R&D contributions to the aims expressed in the vision and spirit of CARS are solicited on a worldwide basis and are reviewed by internationally renowned professionals selected from the IJCARS Deputy

123

Int J CARS Editors and Editorial Board as well as from the IFCARS Congress Organising and Program Committee. (b) The structure of the review process is based on a two tier system whereas long abstracts are reviewed for inclusion in the IJCARS Proceedings Supplement and for presentation at the CARS congress, full papers are reviewed for regular issues of IJCARS, and if applicable (i.e. if requested by the authors), also for presentation at the CARS congress. (c) The assignment of reviewers to submissions is based on their preferred themes and or topics that reviewers have indicated in the CARS ontology for IJCARS and the CARS congress. The end decision of inclusion in regular issues or the proceedings supplement of IJCARS rests finally with the Editor-in-Chief. (d) Those submissions finally accepted for the congress are typically presented in congress tracks of collaborating partners of IFCARS, e.g. ISCAS, CAD, CAR, CMI, IPCAI, EuSoMII, etc. (please refer to individual agreements between IFCARS and the collaborating partners for further details). (e) The assignment of submissions and accepted presentations to the CARS congress tracks and their final timing of presentations, etc. is usually done, in close cooperation with

123

the Chairs/Presidents of the CARS tracks, by the CARS Congress Organizer. This professional collaborative effort is in order to minimize overlap of topics/themes in different CARS sessions of the congress, etc. as well as ascertaining optimal resource allocation of space and time given to the presentations at the chosen conference centre.

General 7. Points 1–6 as outlined above, should be seen as guidelines given by IFCARS as the host to all parties which wish to be collaborating partners by working with IFCARS towards an objective of integrated patient care. These guidelines are open to discussion with the collaborating partners of IFCARS and are subject to change, if and only if, alternative strategies and arrangements, and alternative operational resources can be shown to improve the quality of IJCARS and/or the CARS Congress. They should also be economically viable. Views and suggestions from anyone in the CARS community are also welcomed and may be addressed to the director of IFCARS. Heinz U. Lemke Heidelberg, June 2016

Int J CARS

Computer Assisted Radiology—30th International Congress and Exhibition Chairman: Osman M. Ratib, MD, PhD (CH), Co-Chair: Kiyonari Inamura, PhD (J)

123

Int J CARS Real-time MRI: a new horizon J. Frahm Biomedizinische NMR Forschungs GmbH am MPI fu¨r biophysikalische Chemie, Go¨ttingen, Germany Keywords MRI  Real-time MRI  Interventional MRI  Nonlinear inverse problem Purpose Despite tremendous progress during the past three decades, current MRI techniques are still not able to directly monitor rapid body movements or physiologic processes. For example, cardiovascular MRI studies entirely rely on ECG-synchronized acquisition schemes which extend over several heartbeats and assume myocardial movements and blood flow to be strictly periodic and reproducible. In contrast, this presentation covers a fundamental solution to the basic MRI problem of relatively long measuring times. It describes a novel approach to real-time MRI which yields so far inaccessible image quality and temporal resolution, i.e. MRI movies of arbitrary dynamic processes with image acquisition times of only 10 to 40 ms. The proposed method combines highly undersampled radial gradientecho sequences with serial image reconstruction by regularized nonlinear inversion [1]. Methods Two factors are responsible for the measuring time of a cross-sectional MR image: (1) the time needed for the acquisition of a single spatially encoded MRI signal, i.e. a partial experiment, and (2) the number of partial experiments with different spatial encodings needed for reconstruction of a well-resolved image. The first problem could be solved 30 years ago with the invention of rapid gradient-echo MRI sequences which reduced the repetition times to 2 to 10 ms and for the first time allowed for dynamic imaging [2]. The minimum measuring time of an image decreased from several minutes to about one second which was limited by the second problem, i.e. the use of 128 to 256 partial experiments per image. Thus, the obvious solution for dynamic high-speed MRI is a technique capable of dealing with only very few spatially distinct acquisitions, a concept known as data undersampling. In fact, the proposed real-time MRI method [1] employs radially encoded gradient-echo sequences with up to 40-fold data undersampling, while the image reconstruction emerges as the iterative solution of a nonlinear inverse problem which exploits the similarity of successive frames via temporal regularization. More specifically, real-time MRI may be thought of as a nonlinear extension of conventional parallel MRI techniques which typically offer only twofold undersampling and require knowledge of the coil sensitivities to solve a linear inverse problem. In contrast, the proposed algorithm jointly estimates the desired image together with the sensitivity profiles of all radiofrequency coils that are used in a modern MRI system. For dynamic imaging the resulting ill-conditioned nonlinear inverse problem is markedly improved by temporal regularization to the previous frame. It constrains the range of possible numerical solutions and identifies the ‘‘correct’’ image even for very high undersampling factors. The idea exploits the temporal continuity of physiologic movements in an image series with adequate temporal resolution. Technically, the use of the iteratively regularized Gauss–Newton method ensures the dominance of the data consistency term during numerical optimization. The results not only represent the best possible parallel MRI reconstructions, but also lead to a fully self-consistent technique in the sense that only the actual dataset, but no preceding calibration scan is required for reconstructions of serial images. Online image reconstruction and display is accomplished by a highly parallelized version of the algorithm on a computer equipped with two processors and eight graphical processing units which could be fully integrated into a commercial 3 T MRI system (Prisma Fit, Siemens Healthcare, Erlangen, Germany) [3]. This

123

customized computer is ‘‘invisible’’ to the MRI user: it bypasses the standard reconstruction pipeline of the MRI system, while storing reconstructions as conventional DICOM images in the regular databank. Results The spatiotemporal fidelity of the proposed real-time MRI method has experimentally been validated with the use of a specially designed motion phantom [4]. Preliminary applications range from body movements (e.g., wrist, knee, temporomandibular joint) to studies of the articulators (e.g., during speaking and brass playing) and swallowing (e.g., dysphagia, reflux). Another important area is cardiovascular MRI (Fig. 1), which in future may entirely be based on real-time acquisitions including cardiac function, quantitative blood flow, and tissue characterization using single-shot mapping of the T1 relaxation time. Ongoing research addresses rapid quantitative MRI mapping of a variety of physical (e.g., T1, T2*) and physiological parameters (e.g., flow, perfusion, temperature) preferably by modelbased reconstruction techniques.

Fig. 1 Real-time MRI of the human heart at 33 ms resolution—12 consecutive frames (396 ms) of a single cardiac cycle from (top left) peak systole to (bottom right) early diastole Conclusion This article describes recent developments and first applications of a novel method for real-time MRI which promises superior image quality and spatiotemporal resolution to previous trials. The technique bears tremendous scientific and medical potential and is expected to alter the use of MRI in a number of clinical fields. Apart from cardiovascular MRI, real-time imaging will revitalize ‘‘interventional’’ MRI approaches, i.e. the use of MRI for guiding minimally invasive interventions. References [1] Uecker M, Zhang S, Voit D, Karaus A, Merboldt KD, Frahm J (2010) Real-time MRI at a resolution of 20 ms. NMR Biomed 23:986–994. [2] Frahm J, Haase A, Matthaei D (1986) Rapid NMR imaging of dynamic processes using the FLASH technique. Magn Reson Med 3:321–327 [3] Scha¨tz S, Uecker M (2012) A multi-GPU programming library for real-time applications. In: Algorithms and Architectures for Parallel Processing (Springer). Lect Notes Comp Sci 7439:114–128 [4] Frahm J, Scha¨tz S, Untenberger M, Zhang S, Voit V, Merboldt KD, Sohns JM, Lotz J, Uecker M (2014) On the temporal fidelity of nonlinear inverse reconstructions for real-time MRI— The motion challenge. The Open Med Imaging J 8:1–7.

Int J CARS From computer-assisted radiology to AI in digital imaging C. Peter Waegemann1 Consultant on HIT, Berlin, Germany Keywords Future of CARS  AI Tools that supported both memory and cognitive decision-making enabled human intelligence over thousands of years. The latest knowledge-supporting tools are computers that assist lawyers, architects, accountants, engineers, and hundreds of other professions on a daily basis [1]. Many clinical and administrative computer applications improve clinical and other processes, examples range from computer-assisted medical coding to computer-assisted nursing support. It must be recognized that the scientific body of medicine has grown to a state where no clinician can memorize all the data necessary for daily care functions. Emotions and other distracting features flaw human decision-making. This applies also to radiologists and surgeons. The effects of the information society and the emerging digital society have moved the field of computer assistance to higher levels of sophistication that allow access to more relevant and timely information, as well as specialty-specific algorithms for both text search and image recognition, and natural language processing techniques. Yet how can one integrate all the information onto a meaningful and user-friendly platform? While various approaches, such as TIMSS [2], are being discussed in standards circles, one has to wait for new industry solutions. Yet one must admit the difficulties in creating clinical interoperability during the last decades. In contrast, mobile systems have created interoperability that can integrate thousands of different apps in a secure way. New sensing devices, image recognition techniques, machine learning, use of big data, and the Internet of Medical Things are all part of the new interoperable mobile computing field that has grown as an alternative to traditional health information systems. As connected computers can store both an unprecedented amount of data and algorithms for decision-making, information systems in general have become more system-centric and less dependent on traditional standards and HIS systems. They also provide a new machine intelligence that requires less and less human interaction. Think of automated imaging systems comparing hundred thousands of similar images to promote better understanding. Such a process will be supported by data flowing from other sources, some directly from patients, others from areas that were not traditionally part of the fields of surgery and radiology. More than 300 apps in the field of clinical imaging are used for teaching purposes, as reference assistants, to provide viewing software, to help with patient education, and to assist in various care processes within health information systems. Most of them function outside traditional HIS systems. Also, a wide range of sensing devices that are available today can obtain and transmit patients’ data to the care providers. Yet most hospitals are not equipped to capture such data within their EHRS and imaging diagnostic system. Nor can they arrange the huge volume of data for quick access to relevant information. Image recognition techniques are not quite fully developed, but indications are that such systems will enter the market soon. Such AI-driven systems can identify anatomical features and abnormalities in medical images such as CT, MRI, ultrasound, or other scans and can also draw on text and other data in a patient’s medical record to suggest possible diagnoses and treatments. Much of the image recognition development is currently done behind closed doors of governments, yet once some of these systems will be made public, they can be expected to have a large impact on medical imaging. One example that has gone public with its developments is IBM. Computer-assisted imaging will enter a new era when one adds the functions of machine learning, a process in which systems learn

from derived knowledge based on the volume of images. Big Data will add a new dimension to image processes and in particular to image-based diagnostic processes when previously neglected environmental, patient-specific professional, nutritional, and other nanobot-created information is added. In addition, it is difficult to imagine today how the Internet of Medical Things will affect healthcare and future diagnostic imaging. These are the sensors that are surrounding a patient’s life but are not attached to him/her, such as the bathroom mirror that tracks the patient’s ECG with diagnostic accuracy. Or the toothbrush that looks for cavities and the potential for systemic bacterial contamination that begins in the mouth. Current indications are that those systems will be parallel to traditional HIS systems. The optimal use of new imaging systems is dependent on internal and external communication among all professionals involved in a patient’s care. In order to bring the collective expertise of clinicians and wellness providers to the care process, new communication systems must be created that allow an increased exchange of images and request input from other disciplines. Conclusion The level of sophistication in computer-assistance for medical imaging is increasing to a point where more data will be involved in the process. A new communication pattern should support new AIbased imaging systems in order to provide the best diagnostic imaging intelligence. Decisions should not be made in silos; the radiologist should communicate with others directly involved in a patient’s care. They should enable a surgeon or radiologist to discuss images by exchanging image details as well as EHR data on a routine basis. While the implementation of a communication infrastructure should be priority for all providers, they may also want to look outside the traditional HIS community for new solutions to implement AIdriven solutions. References [1] Waegemann CP (2012) Knowledge Capital in the Digital Society, Amazon. [2] Lemke H (2013) Machine Intelligence in the OR, IHE Online.

Novel user interface concepts for medical image viewing on mobile devices M. Teistler1, B. Schulz1, A. Bischof2 1 Flensburg University of Applied Sciences, Faculty of Information and Communication, Flensburg, Germany 2 University of Luebeck, Department of Radiology and Nuclear Medicine, Luebeck, Germany Keywords Human–computer interaction  Usability  Alternative User Interface  Mobile applications (apps) Purpose Interactive image visualization becomes increasingly important on mobile devices, e.g. for emergency image access or for clinical reference. The user interfaces of the existing mobile applications (apps) for this purpose mainly follow the metaphors known from desktopbased software, usually replacing mouse control with common multitouch gesture control. Examples are scrolling through a stack of images with one finger that is moved forward and backward on the screen and zooming an image with two fingers on the screen. This approach can cause the image content to be obscured by the hand (or a pen) and the image quality to be reduced by a smudged or even scratched display. The goal of this work has been to develop new user interface approaches specific to mobile devices that overcome these disadvantages. In particular, the utilization of inertial sensors has been investigated that can be used to detect the orientation of a mobile device.

123

Int J CARS Methods We developed a medical visualization app for Android tablets, with a special focus on scrolling through a stack of slice images. In this app, a fixed area of the screen is reserved for image viewing without any direct user interaction (Fig. 1a). If the user touches this area, a warning message is displayed. Scrolling is provided to the user in three different ways. The first option is to move a reference line in an interactive scout view (Fig. 1b) for direct selection of images in a specific anatomical region and for scrolling in a way that is typical for touch displays. The second option allows the user to scroll by touching a button on the side of the screen and simultaneously tilting the tablet forward or backward. During this interaction, the difference between the current orientation angle of the tablet and the initial one (when the user started touching the button) determines how many slice images are scrolled forward or backward starting from the initial image (‘‘tilt-based direct scrolling’’). The third option works similarly with the difference that tilting the tablet determines the speed of an automatic scrolling, starting with zero when the interactions starts (‘‘tilt-based cine mode’’) (Fig. 1c).

task was to find three artificial spheres in a CT dataset and to estimate their sizes. The users had to perform this task three times, each time with only one of the three available scrolling options, in random order for each user. In addition, each user performed the task with the freedom of using all scrolling options. For each test step, the time to achieve the given task was measured. The users were observed during the tests and interviewed afterwards. Results In general, the user evaluations have shown that all implemented interaction concepts are considered usable and helpful. The early evaluations revealed that detecting the tilt of the tablet works sufficiently reliable and accurate when only an accelerometer and no other inertial sensors are utilized. Tilt-based scrolling was possible with only small movements of the tablet that could be easily performed by the users. Some users mentioned that the virtual touchpad should be bigger and that the use of the virtual sliders is time-consuming. Overall, the tilt-based direct scrolling was favored over the other scrolling options. Interviews revealed that this feature allows for easy back and forth interrogation of specific anatomical parts. The tiltbased cine mode was considered helpful for systematically reviewing large data sets (like trauma CT), but less useful for a searching task like the one given in the final evaluation. The average times for the tasks in the final evaluation are shown in Table 1. The tilt-based direct scrolling was the fastest mode in average and, except for one user, also the fastest mode for each single user. It was also the mostly used mode when the users were allowed to freely choose between the scrolling options. It could also be observed that the users got accustomed to this mode very quickly. Table 1 Average time required by the users in the final evaluation Scrolling mode

Fig. 1 Basic interaction concepts: Protected image viewing area without direct user interaction (a), interactive scout view for scrolling (b), tilt-based scrolling (c), virtual touchpad (d), virtual sliders (e) An additional option is to switch to a different series within a study by tilting the tablet to the left or to the right. Another option is to change window level and width by tilting the tablet forward or backward and to the left or to the right, resp. For all these tilt-based interactions the user needs to touch a different button on the screen. As a general input feature, a virtual touchpad has been integrated that works like a typical multi-touch pad commonly used as a mouse substitute (Fig. 1d). In addition, virtual sliders are provided at the right and the lower side of the image area that allow the user to define a point of interest within the image, e.g. to define markers or perform simple measurements (Fig. 1e). The app was iteratively developed based on regular informal user evaluations using tablets from multiple vendors. Different clinical cases have been provided for testing, ranging from a head MR data set with only a few slice images to a CT run-off study with over 1000 images. At the end of the development process we carried out a final evaluation with regard to the scrolling options with five test users. The

123

Task Find objects (s)

Estimate size (s)

Tilt-based cine mode

41

65

Tilt-based direct scrolling

26

28

Interactive scout view

37

39

Conclusion New interaction concepts for medical visualization on mobile devices have been developed that can provide a valid alternative to currently implemented concepts. In particular, useful tilt-based interactions can be provided to the user by utilizing a common inertial sensor, particularly for scrolling through a stack of images. Because only small movements of the tablet are required, ergonomic problems can be avoided and the user is not forced to look at the display from significantly different angles which would potentially distort the image impression. As only the most common sensor type is used for the tiltbased interactions, the app consumes reasonable power and can be used on many mobile devices. It is noteworthy that in our evaluations the tilt-based direct scrolling was appreciated more than the interactive scout view that is based on the common touch input paradigm.

Provision of multi-national system for electronic medical record systems

integration

support

B. Crowe1, H. Zhuang1 1 BCA, Canberra, Australia Keywords EMR  Systems integration  Multi-national  China Purpose In China the use of Electronic Medical Record (EMR) systems, including radiology images and reports, has developed on a

Int J CARS departmental basis over the last decade. Hospitals have encountered several obstacles including fragmentation and duplication between diverse software systems; these ‘‘information islands’’ impede data sharing. A two part solution to the problem of data sharing is the availability of an integrated EMR and the availability of experienced System Integrator (SI) EMR support capability. There is extensive experience in the area of EMR SI support in Australia. The 2015 China Australia Free Trade Agreement (ChAFTA) provides an opportunity for the exchange of skills and SI experience in EMR to the benefit of both countries. Methods The technology brief is to develop a successful strategy to localise an overseas EMR products to meet local market requirements in China, supported by a local partner as a Systems Integrator. The new EMR system must be fast and responsive to meet the particular requirements of the need for high throughput of patients in extremely busy hospital departments and outpatient departments in China. As well, the EMR must have the ability to provide a finer granular view to management to identify current bottlenecks and improve profitability and efficiency. A strategy has been developed to establish a dedicated EMR SI capability in Shanghai managed and staffed by experienced bilingual (English/Mandarin) Chinese IT professionals. The SI centre would be based in the Shanghai development park with modern computer hardware, and third party access to secure cloud computing services. The establishment of a dedicated SI service would support hospital IT staff to maintain software with screens in simplified Chinese to ensure that the EMR system is flexible and can be adapted to local hospital requirements. The Shanghai SI centre would be supported by a SI centre in Brisbane Australia with an understanding of the support requirements of EMR. The Brisbane SI centre would act as a repository of expertise and would counter the current excessive staff movement in China. Results The proposed strategy is currently under development. Discussions with prospective staff and partners were held in Shanghai during January 2016 and work is proceeding with the aim of SI implementation during late 2016. Conclusion The Australian and China approach to EMR system integration support is based on an understanding of China health requirements, supported by local knowledge of China’s political imperatives, legislative requirements, and the privacy and security environment essential for delivering cloud based EMR health solutions in China. Experience gained in the establishment of dedicated SI centres for EMR in Shanghai and Brisbane will provide support in China for EMR and achieve the aim of improvements in hospital data sharing.

A VR system for medical simulation using Google Cardboard and Vuforia

technology is bridging the gap between VR and physical reality, many modern developments increase the cost of simulation platforms and make them inaccessible for individuals and institutions. One of the barriers to widespread use of current generation simulators is the high initial cost which is often a result of expensive proprietary hardware [2]. Until recently, these approaches were necessitated; consumer-level VR hardware is only now becoming accessible. A number of low cost head-mounted virtual reality devices are now on the market, with additional devices in development. The Google Cardboard is a low cost system that combines optical lenses with a cardboard shell in order to use mobile phones as the foundation for a head-mounted display (HMD). By itself, Google Cardboard lacks a comprehensive interface for human–computer interaction, especially when considering the requirements for medical and surgical simulations. In this paper, we outline the development of an affordable virtual reality environment for medical simulation making use of a stereoscopic display with head tracking and free form hand interaction. We add positional tracking to Google Cardboard using Vuforia to track an image-based marker that acts as the simulation tabletop platform. Image-based markers are affixed to a pointing tool in order to track a stylus. To evaluate the system, we recruited subjects to perform a simple task involving the localization of points inside ellipsoids within the context of a human head. We analyze the results using a Fitts’s law framework. Methods The requirements of the system included a head mounted stereoscopic display, head tracking (both position and orientation) and a free form spatial input that allows the user to select positions in space using a virtual tool as a 3D cursor as well as trajectories. Google Cardboard was chosen as the ideal platform for our head mounted display due to the low price and accessibility of the product. By default, Google Cardboard provides stereoscopic rendering using the Cardboard SDK as well as orientation tracking using the gyroscope of the embedded mobile phone. While phone gyroscopes are low performance in relation to other head mounted displays, medical simulations don’t require low latency tracking for rapid head movements. In order to add position tracking to the system, we make use of the phone’s camera to track image-based markers that act as the tabletop of the virtual environment. This is enabled through the use of the augmented reality SDK Vuforia. All development is done on a Windows platform using the Unity 5 game development engine. The Android platform was targeted in our initial development. To provide greater freedom in the interface, we used image-based marker tracking to track a stylus with 6 degrees of freedom for pointing and selection tasks. In our initial implementation, a pen is affixed to a flat, double sided marker, which is used to infer the orientation of the pen or the location of the tip. Our initial approach involved the integration of the Leap Motion hand tracking hardware, but mobile processing limitation prohibited effective use. Figure 1 depicts the system in its entirety.

R. Armstrong1, T. Wright1, R. Eagleson2, S. de Ribaupierre3 1 The University of Western Ontario, Biomedical Engineering, London, Canada 2 The University of Western Ontario, Electrical Engineering, London, Canada 3 The University of Western Ontario, Clinical Neurological Sciences, London, Canada Keywords Virtual reality  Surgical simulation  Google Cardboard  Vuforia Purpose Immersive virtual reality (VR) environments are gaining traction in various medical disciplines with applications ranging from teaching anatomy to performing complex surgical tasks [1, 2]. Although the

Fig. 1 A stereoscopic view of the virtual head with the tracked tool. The user can be seen in the upper right, separate from the screenshot

123

Int J CARS The user study was designed as an abstract task to examine targeting performance as well as user perception of usability in order to determine the future direction of development. Many surgical procedures can be decomposed into simplified tasks that require positional targeting or selection of a trajectory within anatomical contexts. In our examination of users’ ability to target structures within the system, we abstracted the targeting of anatomy to generic shapes. Specifically, we examined the performance of users in localizing a point within ellipsoids of various shapes and positions within a transparent human head derived from an existing surgical simulator. A total of 8 participants were recruited through online advertisement. All participants (by self-report) were in good physical and mental health and were free from any impairments that would impact their effectiveness in using our immersive virtual environment. Each participant performed 15 ellipsoid targeting tasks on unique ellipsoids and then repeated the tasks in the same order for a total of 30 point localizations. Ellipsoids were created as graphical meshes using the open source modeling tool Blender 3D. Participants were instructed to maximize accuracy (correctly placing the tool tip within each ellipsoid) while minimizing the task time. For each task, the success of the placement was recorded along with the time. Performance was examined using a Fitts’s law framework [3]. We explored the relationship between target size (volume of the ellipsoid calculated using MeshLab), user accuracy and targeting time. A Likert scale based questionnaire was used in order to obtain qualitative feedback from participants regarding use of the system. The scale is based on work by Witmer and Singer in order to determine user’s sense of presence within the virtual environment [4]. Results For all users, the overall targeting accuracy was 63 % and the average targeting time was 23.8 s. There were no clear relationships between targeting accuracy, task time and ellipsoid size. No clear learning trends were observed. There was high between-user variance, but low intra-user variance for task time, indicating that ellipsoid size and placement did not significantly affect task time, which was more reliant on each user’s individual approach. When examining the data for a speed-accuracy trade-off, no clear trend could be seen. No strong opinions were observed by the questionnaire. On a scale ranging from 1 to 7, where 7 is the most positive outcome, users felt that the sense of objects moving through space was compelling (mean of 5.25), that the input mechanism was natural (mean of 5.13) and that the visual aspects of the environment were immersive (mean of 5.12). Users only somewhat felt that they were able to actively search the environment using vision (mean of 3.63). Additional comments were made regarding the robustness of the tracking system. Some users felt that the interruptions due to marker occlusion disrupted the immersive experience and negatively affected their performance. Conclusion In this study, we were able to develop a cost-effective and portable simulation system with potential for application in surgical training and medical education. The results indicate that the system is usable but requires further developments to improve overall robustness. References [1] Tavakol M, Mohagheghi MA, Dennick R (2008) ‘‘Assessing the skills of surgical residents using simulation,’’ Journal of surgical education, vol. 65, no. 2, pp. 77–83 [2] Chan S, Conti F, Salisbury K, Blevins NH (2013), ‘‘Virtual reality simulation in neurosurgery: technologies and evolution,’’ Neurosurgery, vol. 72, A154–A164 [3] Fitts PM (1954) ‘‘The information capacity of the human motor system in controlling the amplitude of movement’’ Journal of experimental psychology, vol. 47, no. 6, p. 381 [4] Witmer BG, Singer MJ (1998) ‘‘Measuring presence in virtual environments: A presence questionnaire,’’ Presence: Teleoperators and virtual environments, vol. 7, no. 3, pp. 225–240

123

Concepts for IHE integration profiles for communication with probabilistic graphical models M. A. Cypko1, H. U. Lemke1 1 University of Leipzig, Innovation Center Computer Assisted Surgery, Leipzig, Germany Keywords IHE integration profiles  Integrated patient care  Probabilistic graphical models  Decision support system Purpose As a result of the IHE Surgery Kick-off meeting during CARS 2016 in Barcelona and in many subsequent discussions, it is recognized, that for many members within the IHE community, one of the motivational drives to develop IHE integration profiles, is to contribute towards the goal of an integrated patient care (IPC), see Fig. 1.

Fig. 1 Integrated patient care in the context of surgery and its integration with other relevant actors The eight components contributing towards the IPC shown in Fig. 1 may not be representative for health care in general, but may be seen in the context of decision making for patient specific therapeutic approaches including surgery. The health care units which typically play a major role in the process of therapy decision making are specific therapy planning units (TPUs), tumour boards, interventional units, operating suites, etc. Here ideally, many information sources available about the patient (radiology, pathology, oncology, surgery etc.) should be considered before subsequent steps in the diagnostic or therapeutic workflow are being taken. IPC applied in TPUs and operating rooms (OR) as well as related environments can only be achieved if some basic interoperability between all disciplines engaged in the care of a particular patient is being supported [1]. Specifically, communication methods and tools as well as appropriate procedures have to be put into place, such as • • •

Standards for the transmission of data Communication infrastructures Common terminology and understanding (ontologies)

In addition, behavioural agreement of all parties involved with respect to the acceptance and use of patient information structuring and displays as well as interventional workflow models, is a further essential requirement. This type of interoperability would not only benefit the interventional environments but also the hospital or health care organization at large and would enable the introduction of a Clinical Decision Support Systems (CDSS). IHE Surgery has been created to promote the above interoperability features as part of its vision.

Int J CARS Brief vision statement for IHE Surgery: To promote purpose-driven design, modelling, and architecture in the operating room and to ensure communication, knowledge management, safety, and improved outcomes through interoperability. But how can communication infrastructures and procedures be put into place in the health care setting relating to the OR, to achieve something like an IPC with Electronic Medical Record (EMR) enabled access to the right information, at the right place, in the right time, by the right people? Methods One way to realize this concept of an IPC is by (a) leaning on what has already be achieved by the IHE community with existing integration profiles and advancing their adoption while at the same time, and (b) developing specific new integration profiles, for example for therapy planning units, surgical units, etc. It has been challenging to put into practice in the OR many of the new technological and system advances, associated interventional procedures and the corresponding redesign of healthcare infrastructures. An important area of technology development for the Digital OR (DOR) is the IT Infrastructure, e.g. a type of surgical PACS which includes technologies such as DICOM, IHE, EMR and a Therapy Imaging and Model Management System (TIMMS) infrastructure for the storage, integration, processing and transmission of patient specific data including patient models such as multi-entity Bayesian networks (MEBNs). This PACS-like architecture and its application for the management of image and model guided therapy has been the subject of discussions in the DICOM and IHE standard activities. As part of the overall concept ‘‘CDSS based on MEBN’’, in Fig. 2 [2], we present the actors, components and interactions that need to be taken into account for, in the development of a therapy decision support system and integration into the clinical workflow. This concept is in the process of being realised with standards for transmission of data, communication infrastructure and storage for the generic MEBN therapy models and Patient Specific Bayesian Networks (PSBNs), i.e. device and IT system communication between physician’ rooms, tumor boards, TPUs and ORs. These aspects and their standardization play an important role in the integration of CDSS into the clinical workflow and the realization of an IPC.

IHE) and on integration profiles currently being in development (especially in IHE surgery, which itself leans on already established IHE profiles). Related to and extended from suggested integration profiles (IPs) in an IHE Surgery white paper, we suggest and are working on the following exemplary IPs: XX. IHE XD-MMS: Cross-Enterprise Sharing of MEBN Models Summaries XX. IHE XD-PSBNS: Cross-Enterprise Sharing of PSBN Summaries XX. IHE CRPOR: Consistent Representation of PSBNs in the OR XX. IHE CRPTB: Consistent Representation of PSBNs in the Tumor Board XX. IHE CRPWS: Consistent Representation of PSBNs on a Work Station With this initial small list of potential IHE IPs that considers the PSBNs and MEBN therapy decision models, work is progressing towards prototype implementation in selected clinical settings. Conclusion We believe that systems for patient-specific therapy decision support will be vital in IPC. Therefore, it is important to consider standardization work in the early beginning to promote vendor independency in the developments of integrated ORs and their interoperability with connected health care units. Chosen standards for model communication and storage should lean on established standards such as HL7, DICOM, FHIR and appropriate IHE integration profiles. References [1] Lemke HU (2015) Medical device integration in the OR, Interoperability Standards and Issues relating to International Approval Procedures. Health Management Journal. 2015; 15(1):66–73. [2] Cypko MA, Stoehr M, Denecke K, Dietz A, Lemke HU (2014) User interaction with MEBNs for large patient-specific treatment decision models with an example for laryngeal cancer. Int J CARS. 2014;10(1).

The brain–computer interface: nano-hardware and clever software keep CARS on track R. Andrews1 1 World Federation of Neurosurgical Societies, Los Gatos, United States Keywords Brain-computer interface  Deep brain stimulation  Nanoelectrodes  Neuroprostheses

Fig. 2 Concept of a CDSS based on MEBN, partially implemented [2] Results With the use of standards and IHE Integration profiles we aim to support collaborative modelling and validation as well as model interaction. This enables IT systems communication between physician’ rooms, tumor boards, TPUs and ORs. The interoperability aspects between the IT components in Fig. 2 and their standardization play an important role in the integration of CDSS into the clinical workflow and the realization of an IPC. To define standards we lean on established clinical standards, guidelines (e.g. HL7, DICOM, and

Purpose The brain-computer interface (BCI) has relied on noble metal (e.g. platinum) electrodes—from scalp disc electrodes for non-invasive electroencephalography (EEG) to microelectrode arrays embedded in the cortex (e.g. the Utah array). Present-day BCI is limited by poor charge transfer between electrodes and brain tissue, lack of brain chemical (neurotransmitter—NT) monitoring, suboptimal stimulation parameters, and inefficient neuroprosthetic learning routines. Although one widespread clinical application of the BCI—deep brain stimulation (DBS) using platinum macroelectrodes—has proven effective in many patients with movement disorders (e.g. Parkinson’s disease, dystonia), it has been much less effective in other conditions, including epilepsy and mood disorders. Neuroprostheses such as those designed to allow quadriplegic patients to control their own paralyzed limbs (or prosthetic limbs) are handicapped by the limited sampling of brain electrochemical activity as well as by the efficacy of the algorithms linking brain and limb. As the techniques for functional brain imaging (e.g. fMRI) and for understanding the nervous system in animal models (e.g. optogenetics) have become more refined, the need for more precise brain recording and stimulating (both electrical and chemical)—as well as the need for

123

Int J CARS more sophisticated computational analysis tools (software) for guiding the BCI—have become apparent Methods Two major hardware challenges have been how to characterize electrodes (1) to optimize charge transfer (both to maximize the signal on recording and to minimize tissue-damaging electrolysis on stimulation), and (2) to allow real-time measurement of multiple NTs with spatial and temporal precision. Another challenge has been the fabrication of a wireless system for continuous monitoring of brain electrochemical activity in vivo. On the software aspect, several techniques are under development: (1) for BCI devices such as DBS, computational analysis provides stimulation parameters that are much more efficient than present-day high-frequency continuous stimulation [5]; (2) for neuroprosthetic control devices, novel techniques include (a) having the patient teach the neuroprosthesis the correct response (rather than having the patient learn the thought patterns necessary to produce the desired neuroprosthetic response) [2], and (b) dynamical models of single-trial motor cortex activity that will dramatically shorten the learning curve for the patient to be able to control the neuroprosthesis accurately [3]. Results Carbon nanotube/nanofiber (CNT/CNF) arrays coated with conducting polymers (e.g. polypyrrole), result in orders of magnitude increase in capacitance and decrease in impedance to improve charge transfer for electrical recording and stimulating. Novel fast-scan cyclic voltammetry (FSCV) techniques paired with nanoelectrodes allow recording of NT levels—including 2 NT levels simultaneously, dopamine (DA) and serotonin (ST) with ascorbic acid (AA) which is ubiquitous in brain tissue (Fig. 1) [4]. Progress includes: (1) a Bluetooth wireless system for in vivo remote monitoring of NTs during DBS [1], and (2) needle electrodes 1/10 the diameter of present DBS electrodes with 6 nanoarray pads 50 9 20 lm on the tip. Each pad can either stimulate or record electrical activity, or monitor a NT level.

Fig. 1 Top: In vitro DA & ST & AA. Top & Middle: Electrodes: Standard L; CNF R. Bottom: CNF L & R. Top: Individual detection 1 mM AA, 10 mm DA & 10 mm ST, Middle: Ternary mixture, Bottom L: varying ST (10, 5, 2.5, 1, 0.5, 0.25 mm). R: varying DA On the BMI software side, coordinated reset (CR) stimulation for DBS can not only reduce the current needed (extending battery life) but can also reverse overly-synchronized brain firing patterns that underlie disorders from Parkinson’s disease to epilepsy. In Parkinsonian monkeys, CR stimulation can reduce Parkinsonian symptoms for up to a month after the stimulation is turned off—a remarkable finding given that in standard DBS symptoms return in minutes

123

(Fig. 2) [5]. In the ‘‘teaching’’ rather than ‘‘learning’’ BCI paradigm, the user evaluates responses of the neuroprosthesis as either correct or incorrect [2]. The reinforcement learning algorithm in the neuroprosthesis takes the user’s correct versus incorrect response to quickly learn the appropriate motor behaviors. The neural dynamics technique has been used in macaque BCI studies to improve performance on single trials by ‘‘denoising’’ the data, allowing more accurate predictions of performance on future trials than otherwise possible [3].

Fig. 2 CR & DBS in MPTP monkeys. Y-axis = locomotor activity. Under standard high-frequency DBS stimulation, effects are lost shortly after discontinuation of stimulation; under CR stimulation, the effect is maintained for over 30 days after discontinuation of stimulation Conclusion Thanks to ‘‘nano-hardware’’ the ability to monitor both electrical activity and multiple NT concentrations in vivo with hitherto unavailable spatial and temporal resolution will add significantly to our knowledge of brain function in both normal and diseased states. The information gained from such electrochemical monitoring can be used to guide the DBS more efficiently (feedback-guided stimulation). Together with enhanced ‘‘software’’ such as the teaching BCI paradigm and single-trial dynamical modeling, the future of the BCI—a prime example of computer-assisted surgery—for both neuormodulation and neuroprostheses is bright indeed. References [1] Chang SY, Kimble CJ, Kim I, et al. Development of the Mayo Investigational Neuromodulation Control System: toward a closed-loop electrochemical feedback system for deep brain stimulation. J Neurosurg 2013; 119:1556–65. [2] Iturrate I, Chavarriaga R, Montesano L, et al. Teaching brainmachine interfaces as an alternative paradigm to neuroprosthetics control. Sci Rep 2015; 5:13893. [3] Kao JC, Nuyujukian P, Ryu SI, et al. Single-trial dynamics of motor cortex and their applications to brain-machine interfaces. Nat Comm 2015; 6:7759. [4] Rand E, Periyakaruppan A, Tanaka Z, et al. A carbon nanofiber based biosensor for simultaneous detection of dopamine and serotonin in the presence of ascorbic acid. Biosens Bioelectron 2013; 42:434–38. [5] Tass PA, Qin L, Hauptmann C, et al. Coordinated reset has sustained after effects in Parkinsonian monkeys. Ann Neurol 2012; 72:816–20.

New method for the automatic retrieval of radiological records based on Radlex annotations A. Spanier1, D. Cohen1, L. Joskowicz1 1 The Hebrew University of Jerusalem, Computer Science and Engineering, Jerusalem, Israel

Int J CARS Keywords CBIR  Radlex  Retrieval  Radiology Purpose The increasing amount of medical imaging data acquired in clinical practice constitutes a vast database of untapped diagnostically relevant information. To date, only a small fraction of this information is used during clinical routine for research and population studies due to the heterogeneity, complexity, high dimensionality, and difficulty of locating relevant studies in the vast database. Radiologists and clinicians are increasingly struggling under the burden of diagnosis and follow-up of this collection of scans. Medical Content-Based Image Analysis and Retrieval (M-CBIR) techniques have been proposed to tap into this vast database by providing tools for identifying relevant cases based on their image similarity and to provide support to radiologists in the clinical decision process. In addition to images, the radiology case reports include textual information on the anatomy and pathology findings of the patient. This textual information, when described in a standardized terminology such as the RadLex lexicon, constitutes a substantial source for image retrieval systems. It can be collected from radiologists reports or automatically derived from the clinical images. We have developed a new method for the retrieval of similar radiological cases based on the Radlex, a publicly available lexicon that provides a uniform standard for all radiology-related information. RadLex consists of a hierarchy of over 68,000 terms encompassing many domains, ranging from basic sciences to imaging technologies, and acquisition. This paper introduces the Augmented RadLex Graph (ARG), a similarity distance metric (ARG-SIM) that ranks medical cases in this ARG representation, and a method for accurately and efficiently quantifying cases similarities and retrieve the most similar clinical cases. Methods The input to our method is a list of Anatomy-Pathology Radlex Terms (APRT) terms describing the radiological finding of a given case and an annotated database of clinical cases. The output is an ordered list of the 10–30 most relevant cases in the database. The Augmented RadLex Graph (ARG) is built from the generic RadLex hierarchical tree by augmenting it with cross-tree edges connecting anatomical structures to pathologies in the case APRT list (Fig. 1). The ARGSIM function computes the weighted link distance in ARG between the APRT pairs. The function ARG-SIM is designed so that closely matching cases with more specific characterizations are scored higher than those that are more general and less detailed.

Fig. 1 Illustration of the ARG-SIM: The bold black edges show the added edges of the query case on top of the RadLex tree (the ARG). The dashed red arrows show the link distance between the APRT pairs of query cases to the database case. The first APRT is identical, so its link list is one. The second APRT has the same anatomy— Liver, but a more specific pathology so its link distance is 2. Thus, the overal similarity is 1/3— normalized into a range of [0,1)

The case retrieval method starts by building the ARG for the given query case. Next, each case in the database is compared to the query case using the ARG and assigned a similarity metric with ARG-SIM. The resulting scores are then sorted in descending ARG-SIM values and the top 10 (or 30) are selected. Results We evaluated our method on the VISCERAL Retrieval Benchmark database. The database consist of 2,311 volumetric scans acquired with various scanning protocols, e.g., CT, MRI-T1, and MRI-T2. The scans were acquired as part of the daily clinical routine work from three different clinical centers. The benchmark includes 8 CT test cases of different pathologies. The retrieved cases where the evaluated by a radiologist, who performed relevance assessment of the top ranked cases. For each test case, the evaluation measures of the 10 and 30 most relevant cases were reported. The evaluation measure is defined as the proportional number of cases from the list that are deemed relevant by the medical experts. We compared our new results to the published ones of three methods [1–3] from the 2015 Visceral Challenge. Figure 2 shows the results of our method and that of the others. Note that our consistently scored as the best one.

Fig. 2 Scores of our method and that of three other methods from the 2015 Visceral Benchmark for the top 10 and top 30 cases Conclusion We have presented a new method for retrieval of medical cases based on their textual clinical reports. The advantages of our method are that it incorporates both image and textual information, that it creates a standardized representation for query cases, and that it allows quantitative comparison between cases of varying degrees of detail and different pathologies in different organs. Our method is fully automatic and requires no user intervention. Our preliminary results on the VISCERAL Challenge benchmark indicate that the method is practical and achieves a good scoring in most cases. References [1] Jimenez-del-Toro OA, Cirujeda P, Dicente Cid Y, Muller H (2015) RadLex Terms and Local Texture Features for Multimodal Medical Case Retrival. In: Multimodal Retrieval in the Medical Domain. Lecture Notes in Computer Science, Vol. 9059. Springer (2015). [2] Choi S (2015) Multimodal Medical Case-Based Retrieval on the Image and Report: SNUMedinfo at VISCERAL Benchmark. In: Multimodal Retrieval in the Medical Domain. Lecture Notes in Computer Science, vol. 9059. Springer (2015). [3] Zhang F, Song Y, Cai W, Depeursinge A, Muller H (2015) USYD/HES-SO in the VISCERAL Retrieval Benchmark. In: Multimodal Retrieval in the Medical Domain. Lecture Notes in Computer Science, Vol. 9059. Springer (2015).

123

Int J CARS A PACS-oriented multimodal search engine E. Pinho1, F. Valente1, C. Costa1 1 IEETA, DETI - University of Aveiro, Aveiro, Portugal Keywords Multimodal IR  Medical imaging  CBIR  Query fusion Purpose The use of digital medical imaging systems in healthcare institutions has increased significantly, and the large amounts of data in these systems have led to the conception of powerful support tools: the continued efforts in image processing, medical informatics and, in recent times, multimodal information retrieval are creating conditions for their integration in radiology workflows [1]. On the other hand, the subject is still under heavy research, and very few solutions have become part of Picture Archiving and Communication Systems (PACS) in hospitals and clinics. This paper proposes an extensible platform for multimodal medical image retrieval. Multimodality, in this context, refers to the available information in medical imaging repositories, such as medical image meta-data, pixel data, structured reports and similarly annotated content. It is fully integrated with Dicoogle [2], an open-source PACS archive, and relies on an existing CBIR platform [3]. The result is a flexible architecture for executing multimodal searches in a PACS, as well as an assortment of web services for use by humans and other programs. Multimodal search capabilities were integrated with a new Dicoogle plugin, contemplating the following goals: (1) Create an interoperability layer among different information sources, namely text-based, image-based query providers and potentially other information modalities in the future. (2) Integrate state of the art query fusion techniques and leverage the potential of Dicoogle’s CBIR plugin so as to be put in image retrieval benchmarking scenarios and in clinical practice. (3) Exploit a flexible and usable search user interface relying on state of the art paradigms in the field, such as query-by-example. Methods The two entry points for multimodal queries are a RESTful API and a user interface. The multimodal search engine processes the queries and, depending on the kind of query requested, makes calls to the appropriate Dicoogle query providers through an interface adapter. The multimodal retrieval engine currently contemplates two modality interfaces of the Dicoogle platform. DICOM-based text query processing is based on a Lucene index [2]. Image queries, on the other hand, are processed by the Dicoogle CBIR plugin [3]. The proposed system features multimodal queries based on a combination of multiple text and image content objects forming a tree structure of queries, where a leaf of the tree represents a unimodal query and other tree nodes represent query fusions. An assortment of late query fusion strategies such as CombSUM, CombMNZ and RRF were included, and scores are automatically normalized when necessary [4]. When a multimodal query is received, the server may apply a fixed series of transformations to the query. Afterwards, each unimodal query is invoked on the Dicoogle core runtime, yielding multiple result streams. These lists are combined into a single list through late query fusion. The process ends with a series of transformations over the list, such as DICOM attribute injection and DICOM Image Model aggregation. Lastly, the system provides a web-based graphical user interface. Each sub-query in the tree is represented as a box. Images from a previous result or a local file can be dragged and dropped over a query box in order to become part of the query. Text queries are still supported by typing on a text input box. Query fusions are made by dropping targets in a highlighted region below a query box, allowing the user to choose a particular fusion strategy or let the engine decide automatically.

123

Results With this work, we have designed and built an extension to Dicoogle for multimodal search capabilities. Its backbone was designed to be extensible with new algorithms without major changes to the software, thus being adequate for testing new retrieval algorithms over existing benchmarks, as well as for being put to clinical practice. The features mentioned in this abstract were implemented and a public demo is available at the official Dicoogle website. Conclusion The outcomes described here are still admitted as a foundation for future work that will hopefully further narrow the semantic gap of medical image/case retrieval and facilitate its way to becoming effectively and efficiently used in practice. The evaluation of retrieval performance was not covered by this contribution, as such a validation would demand the specification of a concrete set of unimodal medical image retrieval techniques. Rather, we have designed and implemented an architecture providing a multimodal layer of abstraction over the large domain of existing retrieval algorithms, such as feature extractors and model representations. Nevertheless, the validation of a complete solution, as well as the introduction of new ways to exploit the available information in a medical imaging archive, will be addressed in the near future. References [1] Akgu¨l CB, Rubin DL, Napel S, Beaulieu CF, Greenspan H, Acar B (2011) Content-Based Image Retrieval in Radiology: Current Status and Future Directions,’’ Journal of Digital Imaging, vol. 24, no. 2, pp. 208–222. [2] Costa C, Ferreira C, Bastia˜o L, Ribeiro L, Silva A, Oliveira JL (2011) Dicoogle—an open source peer-to-peer PACS, Journal of digital imaging, vol. 24, no. 5, pp. 848–856. [3] Valente F, Costa C, Silva A (2013) Dicoogle, a PACS featuring profiled content based image retrieval, PloS one, vol. 8, no. 5, p. e61888. [4] Depeursinge A, Mu¨ller H (2010) Fusion Techniques for Combining Textual and Visual Information Retrieval, in ImageCLEF, vol. 32, H. Mu¨ller, P. Clough, T. Deselaers, and B. Caputo, Eds. Springer Berlin Heidelberg, pp. 95–114.

Patch based codebook model for focal liver lesion retrieval of multiphase medical volumes J. Wang1, X.- H. Han1, Y. Xu2, L. Lin2, H. Hu2, C. Jin2, Y.- W. Chen1 1 Ritsumeikan University, Graduate School of Information Science and Engineering, Kusatsu, Japan 2 Zhejiang University, Hangzhou, China Keywords Focal liver lesion  Codebook learning  Multiphase medical images  CBIR Purpose The content based medical image retrieval plays an important role in medical diagnosis and experience sharing. In order to retrieval similar cases to a query sample, the extraction of inherent feature representation for medical images is highly needed. Roy et al. [1] and Xu et al. [2] used intensity and texture features in representation of medical images. This study explores a codebook-based feature representation, which has already been proved to be extremely effective for classification and retrieval of natural images. The proposed codebook model uses patches as local descriptors and the K-means algorithm is employed. The proposed method is applied to the retrieval of 5 types of focal liver lesions using multiphase medical volumes. Experiments show that promising retrieval performance can be achieved.

Int J CARS Methods The proposed codebook model can be constructed in four steps: (1) Local feature extraction, where we use raw patches Y = [y1, y2,… yN], N is the total number of patches, as local descriptors; (2) Codebook learning based on K-means algorithm. The K-means algorithm learns the representative patches set, i.e. codebook B, for best representing the training patches and simultaneously gives the coded vector xi, i = 1,2,…,N, for local descriptors yi by solving the following objective function: argminjjY  BXjj2 ; jjxi jj0 ¼ 1; jjxi jj1 ¼ 1; xi  0: where X = [x1, x2,… xN] can be considered as the coded vector for Y, and the first constraint limits only one codeword is used for representation of one input patch and the second constraint restricts the weight to be one; (3) coding local feature according to the learned codebook. The corresponding element in the coded vector to its nearest codeword is 1, others are 0 s. (4) Integrating all the coded vectors from the same medical image as a final feature vector for image representation is called the mean pooling procedure. We separately learn codebooks for all three phases medical volumes and obtain a feature vector for each phase. The feature representation of the multiphase medical case is the concatenation of feature vectors from all three phases. With the obtained feature vectors, we simply uses Euclidian distance for retrieval of most similar cases to the query sample. Results To evaluate the performance of the proposed method, we construct a dataset of focal liver lesion cases with the help of radiologists. The dataset consists of five types of focal liver lesions and in total 137 medical cases. Each of the cases includes 3 phases’ medical volumes: non-contrast enhanced (NC) phase, obtained before contrast injection; artirial (ART) phase taken 25–40 s after contrast injection and portal venous (PV) phase, which is performed 60–75 s after contrast injection. Figure 1 shows examples from each of the five types of liver lesions and the columns illustrate the visual difference on different phases.

Fig. 2 Quantitative evaluation of experiment results Conclusion In this study, a codebook model for retrieval of multiphase medical volumes is presented. We used raw patches as local descriptors and applied K-means for codebook learning and coding procedure. Experiment results show that promising performance can be achieved in retrieval of multiphase medical volumes. However, in conventional coding process, such as K-means, we need to unfold three-dimensional patches into vectors, which will destroy the inherent sptatial structure in 3-D data. Hence, in future work, we are going to explore multilinear techniques for directly coding 3-D data without unfolding, which is prospected to give further improvements in multiphase medical volumes retrieval. References [1] Roy S, Chi Y, Liu J, Venkatesh SK, Brown MS (2014) Threedimensional spatiotemporal features for fast content-based retrieval of focal liver lesions. IEEE Trans Biomed Eng 61(11):2768–2778. [2] Xu Y, Lin L, Hu H, Yu H, Jin C, Wang J, Han X, Chen Y (2015) Combined Density, Texture and Shape Features of Multi-phase Contrast-Enhanced CT Images for CBIR of Focal Liver Lesions: A Preliminary Study. Innovation in Medicine and Healthcare 2015 45:215–224.

Development of a workflow-oriented structured report in wound care

Fig. 1 Examples of each lesion type on 3 phases In order to evaluate the performance of our proposed method, we use precision and recall as quantitative measure matrix and the precision versus recall curve is plotted. The performance of the proposed method compared with those used in Ref. [1] is shown in Fig. 2. Top retrieval results for five query lesions, one from each of the five lesion classes, are also illustrated in Fig. 2. The proposed multiphase codebook model has the best performance among all features, especially when only take several most similar cases to supply to doctors for reviewing, which is the common way in reality and medical practices.

M. Apitz1, K. Kinzel2, R. Pahontu2, O. Heinze2, B. Bergh2, B. P. Mu¨ller1, H. G. Kenngott1 1 Heidelberg University, General, Visceral and Transplantation Surgery, Heidelberg, Germany 2 Heidelberg University, Department of Medical Informatics, Heidelberg, Germany Keywords Google Glass  Surgery  Wound  Wearable computer Purpose Postoperative wound infections are among the most common postoperative complications in general surgery. With the increase of diabetes mellitus, vascular and cancer diseases in the elderly people, incidence of wound complications and the demand for documentation and visitation might increase. Since wound procedures are mostly septic, a standardized report using a wearable and voice-commandable data goggle was developed for visualization and documentation of patient data.

123

Int J CARS Methods The software ‘‘WoundTrack’’ was developed for integrating and visualizing patient data on the Google GlassTM (Google Inc., USA) (Fig. 1). The software provided a structured report, which reduced the patient history to relevant information. To evaluate the system, surgical residents were enrolled in a study, had to perform specific tasks and were presented a standard questionnaire. Residents had to assess silicone wound phantoms created from 3D scans of real wounds (Fig. 2), decide for a therapy and document them with a photograph. Patient history was developed from anonymized patient data. The WoundTrack software was compared to the documentation process on a desktop PC, as it is the status quo of most surgical clinics. The time to perform the tasks were measured. Participants were asked to grade work contentment on a scale from 1 (best) to 6 (worst). Furthermore, provision of information, hardware and software features were evaluated through a Likert scale (1 ‘‘strongly agree’’ to 4 ‘‘strongly disagree’’).

Fig. 1 Example of Report Section of the WoundTrack Software

Fig. 2 Example of the Wound Moulages Results 20 surgical residents and surgeons were included. 75 % (15/20) of participants had diagnostic and therapeutic experience with [ 50 wounds. 70 % (7/10) in the intervention group had no experience with the Google GlassTM. All participants (20/20) certified subjective realism of the test environment. With the WoundTrack software, time to complete the requested task was not significantly different (median 20 min vs. median 19 min with the Google GlassTM), though the participants using the glass felt their work faster (10/10). The software led the user through the visitation according to their workflow (10/10). Grading of work contentment did not differ, since contentment in both groups was rated ‘‘good’’ The rate of informed decision-making did not differ significantly. The voice control was fluent (10/10), whereas comfort of the frame (50 %, 5/10) and visibility of the display were impaired (60 %, 6/10), more in those wearing spectacles.

123

Conclusion In terms of working time and contentment, the WoundTrack software was equivalent to the current desktop documentation. Controlling the software by speech commands was successful and might accelerate documentation while performing septic procedures of wounds. However, technical and hardware issues of the Google GlassTM are the strongest impediments against the clinical introduction and have first to be overcome. Telemedicine national framework for healthcare and education S. Mishra1, R. Chand2, I. Singh2, V. Singh2 1 Sanjay Gandhi Post Graduate Institute of Medical Sciences, Dept of Endocrine Surgery, Lucknow, India 2 School of Telemedicine & Biomedical Informatics, Lucknow, India Keywords Telemedicine  eHealth  Mobile health  Tele-Education Purpose This descriptive paper aims at informing participants on the landscape of telemedicine across the Indian sub-continent covering government national policy, technology infrastructure including national high speed internet backbone, Telemedicine activities across the nation, Indian projects overseas etc. India being a developing country needs technology for its social sector development. In that context government policy on Digital technology promotion holds a great deal of promise for the industry to develop Information Technology enabled products and solutions. Also, the rapid development of mobile network has brought in new trends in health apps as a business proposition by startups. Methods The report has been prepared compiling information on country wide activities through various sources like conference proceedings, peer reviewed publications, various government meeting proceedings in which the author is a technical committee member, personal communication with peers and electronic resources. Since the first telemedicine experiment in 1999 India has gone through several phases of development to master the technology and been able to undertake national programs besides institutional and corporate activities around the country [3]. To start with most telemedicine activities were in the project mode mainly supported by federal grants from Indian Space Research Organization [2] and Department of Information Technology, Ministry of Communication and IT. These projects helped in developing indigenous technology, software, systems and standards. Further, the pilot projects made the people aware besides testing the relevance of technology for Indian health system [1]. However, in the year 2005 the Ministry of Health & Family Welfare (MoH&FW) constituted a National Task Force for Telemedicine which became instrumental in framing policy guidelines. For the first time budget was allocated for e-Health including Telemedicine during 11th Five Year Plan (2007–2012). India has now acquired rich experience in implementing large number of telemedicine projects over one and half decade. There has been a revolution in fast adoption in mobile communication technology in recent years further driving mobile health market. A few corporate hospitals have developed their own telemedicine network including provision of transcontinental tele-radiology and medical transcription services. Case study: National Medical College Network (NMCN) MoH&FW, Government of India is launching a countrywide networking of all medical colleges to facilitate tele-education/e-learning, access to specialist consultation and access to electronic knowledge repositories [4]. This infrastructure would facilitate continued professional skill development of human resources for health (HRH) [5]. The proposed national network of medical colleges will follow a hierarchical architecture having a Central HUB housing the data center which will be designated as National Resource Center (NRC). The NRC will be networked with Five Regional Resource Centers (RRCs) located at different regions of the country which in turn will

Int J CARS be networked with Medical Colleges in that region. The School of Telemedicine & Biomedical Informatics (STBMI) at Sanjay Gandhi Postgraduate institute of Medical Sciences (SGPGIMS), Lucknow has been identified as the NRC and Central HUB. Five RRCs to be located at Postgraduate Institute of Medical Education & Research (PGIMER), Chandigarh; All India Institute of Medical Sciences (AIIMS) New Delhi; SGPGIMS, Lucknow; North Eastern Indira Gandhi Regional Institute of Medical & Health Sciences (NEIGRIMHS), Shillong; Jawaharlal Nehru Institute of Postgraduate Medical Education and Research (JIPMER), Pudducherry and King Edward Memorial (KEM) Medical College, Mumbai. These RRCs will be networking each with medical colleges in that geographical region. The network will be based on Giga byte internet fibre bandwidth deployed by National Knowledge Network (NKN). Results Ministry of Health & Family Welfare has implemented few projects nationwide such as Integrated Disease Surveillance Project (IDSP) in 2007, National Cancer Network (ONCONET) in 2009, National Rural Telemedicine Network (2009), Digital Medical Library Network (2009), National Knowledge Network (2010), and National Medical College Network (www.nmcn.in) in 2014. Telemedicine standardization and practice guidelines were developed way back in 2003 by the Department of Information Technology in the Government of India. The External Affairs Ministry has taken up the Pan-African Telemedicine Network Project and the SAARC Telemedicine Network Projects in the year 2010. National Knowledge Network (www.nkn.in), a federal government initiative since 2010 has enabled all government educational institutions to get access to high speed internet bandwidth for collaboration and knowledge/skill sharing. Smart Lecture theatres were established in these institutions and National Digital Library was launched under University Grant Commission funding. A National Resource Centre (www.nrct.in) has been established at Lucknow besides five Regional Resource Centers in each geographical region of the country to steer national medical college network. School of Telemedicine & Biomedical Informatics (www.stbmi.ac.in) was established at Sanjay Gandhi Post Graduate Institute of Medical Sciences (SGPGIMS), Lucknow with the financial support from provincial and federal government which is the only academic center to offer one year diploma courses in five streams of Health IT besides offering training opportunities to WHO sponsored fellows from the region. Conclusion This report has covered all round development of telemedicine across the Indian nation, a vast country with diverse geography, culture, language and socio-economic and political complexity. It is worth watching this technology development and adoption by the society where the healthcare access is still not reaching remote areas. Technology like mobile connectivity and hand held tools holds great promise to meet the Sustainable Development Goals (SDG) formulated recently by the United Nations. India is acquiring a sizeable market segment in health care BPO (business-process outsourcing) and KPO (knowledge-process outsourcing) industries. It is now preferred as a healthcare destination in the region, so telemedicine is going to play a major role in promoting medical tourism in time to come. With the rapid expansion of mobile wireless broadband deployment, India will rip the benefit of mHealth in providing access to health for its rural population in particular to patients suffering from non-communicable disease and geriatric health problems. References [1] Ayyagari A, Bhargava A, Agarwal R, Mishra SK, Mishra AK, Das SR, Shah R, Singh SK, Pandey A. Use of telemedicine in evading cholera outbreak in Mahakumbh Mela, Prayag, UP,India: an encouraging experience. Telemed J E Health 2003;9(1):89–94. [2] Singh K, Mishra SK, Misra R, Gujral RB, Gupta RK, Misra UK, Ayyagari A, Basnet R, Mohanty BN. Strengthening Postgraduate Medical Education in Peripheral Medical Colleges through Telemedicine. Telemed J E Health 2004;10:S 55–6.

[3] [4]

[5]

Kapoor L, Mishra SK, Singh K, ‘‘Telemedicine: experience at SGPGIMS, Lucknow. J Postgrad Med. 2005 Dec;51(4):312–5. Mahapatra AK, Mishra SK, ‘‘Bridging the Knowledge and Skill Gap in Healthcare: SGPGIMS, Lucknow, India Initiatives’’ Journal of eHealth Technology and Application. 2007 June;5(2):67–69. Mahaparta AK, Kapoor Lily, Singh Indra Pratap, Chand Repu Daman, Mishra SK.’’Capacity Building in e-Health in a Developing Country—Indian Initiatives’’. Journal of eHealth Technology and Application. 2008 July; Volum6 (1):61–62.

Real-life experience with the e-ASPECTS software in a tertiary stroke center: report on the first 100 acute ischemic stroke cases S. Nagel1, S. Scho¨nenberger1, J. Pfaff2, J. Purrucker1, C. Herweh2 1 University Hospital Heidelberg, Neurology, Heidelberg, Germany 2 University Hospital Heidelberg, Neuroradiology, Heidelberg, Germany Keywords Computed tomography  Ischemic stroke  ASPECTS  Machine learning Purpose e-ASPECTS is a CE-marked software and is intended to aid the physician in the acute management of ischemic stroke patients. It assesses ischemic damage on a non-contrast enhanced head CT (NECT) by applying the Alberta Stroke Program Early CT Score [1,2] (ASPECTS; range 0 to 10, with lower scores indicating more early ischemic changes in the territory of the middle cerebral artery, Fig. 1). It has previously been shown that e-ASPECTS has superior performance to junior stroke physicians and is equivalent to expert neuroradiologists. It uses 3D registration of the NECT and has a segmentation and a scoring module, which is based on a machine learning algorithm. We have been using e-ASPECTS in our routine clinical stroke imaging pathway.

Fig. 1 Alberta Stroke Program Early CT Score

123

Int J CARS Methods e-ASPECTS is installed on a local server. NECT DICOM images from a 64-slice SOMATOM Definition AS CT scanner (Siemens AG, Healthcare Sector, Forchheim, Germany) are pushed manually on demand to e-ASPECTS, automatically analysed and the output is available via intranet based web interface and email alerts (Fig. 2). Of the first consecutive 100 patients with suspected acute ischemic stroke that have been analysed with e-ASPECTS in a real life scenario, we descriptively assessed the performance of the software. In 86 cases a CT perfusion and/or angiography was available for comparison. In addition, a structured questionnaire was completed by nine stroke physicians who have been regularly using e-ASPECTS (only four authors participated).

[2]

damage on brain computed tomography scans: E-aspects. European Medical Journal;3(1):69–74. Herweh C, Ringleb PA, Rauch G, Gerry S, Behrens L, Mo¨hlenbruch M, et al. (2015) Performance of e-ASPECTS software in comparison to that of stroke physicians on assessing CT scans of acute ischemic stroke patients. International Journal of Stroke (in press, accepted 12.Nov. 2015).

Comparison of measurement tolerance to CT radiation dose reduction on CADv among three different reconstruction algorithms in phantom study Y. Ohno1, A. Yaguchi2, T. Okazaki2, K. Aoyagi3, H. Yamagata3, H. Koyama4, K. Sugimura4 1 Kobe University Graduate School of Medicine, Division of Functional and Diagnostic Imaging Research, Department of Radiology, Kobe, Japan 2 Toshiba Corporation, Corporate Research and Development Center, Kawasaki, Japan 3 Toshiba Medical Systems Corporation, Otawara, Japan 4 Kobe University Graduate School of Medicine, Division of Radiology, Department of Radiology, Kobe, Japan Keywords Lung  Computer-aided volumetry  CT  Radiation dose

Fig. 2 e-ASPECTS setup within the hospital network Results e-ASPECTS was able to process all NECT. Results were available within 2 min after push. Median e-ASPECTS was 9 (2–10, min– max) and in 43 patients a score of 10 was given. 55 patients were treated with thrombolysis and/or thrombectomy. In 84 cases the software correctly identified the ischemic side or a score of 10. In 94 patients the e-ASPECTS output guided or was in line with the clinical decision making and in only 6 patients e-ASPECTS made clinically relevant misclassifications (e.g. large infarct on NECT not detected, infarct detected that was not present). All physicians said that e-ASPECTS increases their confidence when interpreting NECTs and 8 of 9 said that it helps them making treatment decisions faster. Overall, all survey responders stated good or excellent experience with e-ASPECTS and would like to continue using it. Conclusion e-ASPECTS is a valuable tool in the clinical pathway of acute ischemic stroke patients and can give a standardised and unbiased assessment of NECTs to assist treatment decision making. References [1] Hampton-Till J, Harrison M, Ku¨hn AL, Anderson O, Sinha D, Tysoe S, et al. (2015) Automated quantification of stroke

123

Purpose The Quantitative Imaging Biomarkers Alliance (QIBA) established by RSNA has evaluated the measurement accuracy of computer-aided volumetry (CADv) software provided by many vendors for a QIBA recommended phantom study [1]. In addition, all CT suppliers have incorporated hybrid type iterative reconstruction and/or model-based iterative reconstruction methods for clinical use to further reduction of the CT examination radiation dose compared with the filtered back projection (FBP) method, and several investigators have reported on their respective clinical utilities in the last few years [2–4]. Recently, the forward projected model-based iterative reconstruction (FIRST) method was developed as a model-based iterative reconstruction technique by Toshiba Medical Systems Corporation. The purpose of this study was thus to directly compare the measurement tolerance of CADv to CT radiation dose reduction among three different reconstruction algorithms including FIRST, AIDR 3D and FBP in QIBA chest phantom study. Methods Multiple layouts of synthetic nodule combinations were placed within the vasculature insert of an anthropomorphic thorax phantom (Chest Phantom N-1 LUNGMAN; Kyoto Kagaku Co., Ltd., Kyoto, Japan). The simulated spherical nodules included five different sizes (3, 5, 8, 10, and 12 mm), three radiodensities: -800 (ground-glass nodule [GGN] phantom), -630 (part-solid nodule [PSN] phantom), and +100 HU (solid nodule [SN] phantom). In addition, two simulated nodules of each type were positioned in the peripheral lung zones within the phantom. In most layouts, nodules were directly attached to the vasculature and/or chest wall using radiographically lucent doublesided tape for both lungs. All CT data were obtained with a 320-detector row CT scanner (Aquilion ONE; Toshiba Medical Systems, Otawara, Tochigi, Japan). The tube currents used in this study were 270 mA, 200 mA, 120 mA, 80 mA, 40 mA, 20 mA and 10 mA. CT examinations were performed three times at each tube current, and reconstructed by FBP, AIDR 3D and FIRST methods. All measurements were performed by a commercially available workstation (Vitrea; Vital Images, Inc., Minnetonka, MN), and evaluated by means of automatic three-dimensional (3D) volumetry software (CT Lung Nodule Analysis; Vital Images, Inc.).

Int J CARS To determine the utility of FIRST for CT examination with a lower dose than for AIDR 3D and FBP, image noise within each nodule was determined by using ROI measurements. To determine whether FIRST was more effective for improving the capability of automated volumetry than other methods, each simulated nodule was measured with CADv software. To compare the capability of the three methods for image quality improvement for each nodule type group, the mean image noise at each tube current associated with each of the three methods was compared by means of Tukey’s HSD test. To compare the capability of CADv at each tube current for each of the nodule type groups, the percentage of absolute measurement errors for all reconstruction methods were also compared by means of ANOVA followed by Tukey’s HSD test. Finally, to compare the capability of AIDR 3D and FIRST methods with standard-dose CT for radiation dose reduction while maintaining measurement accuracy of CADv for all nodule type groups, the percentage of absolute measurement errors was compared by means of ANOVA followed by Dunnett’s test for CT images obtained at each tube current and reconstructed with AIDR 3D and FIRST, and that obtained at 270 mA and reconstructed with FBP. Results Results of a comparison of image noise for the three methods and each nodule type group are shown in Table 1, showing that mean image noise of FIRST for each nodule group was significantly lower than that for AIDR 3D and FBP at each tube current (p \ 0.05). Mean image noise of AIDR 3D for each nodule group was also significantly lower than that of FBP at each tube current (p \ 0.05). Table 1

Table 2

GGN(-800 HU)

Reconstruction method

FBP

AIDR 3D

14.0 ± 8.9

11.9 ± 7.1

9.3 ± 6.3

7.2 ± 4.7

6.0 ± 4.6

5.9 ± 4.2

7.0 ± 6.1

7.0 ± 5.6

5.6 ± 5.1

14.8 ± 12.9

13.3 ± 9.5

10.3 ± 6.7

8.6 ± 12.9

7.1 ± 5.2

5.8 ± 6.3

4.7 ± 10 4

7.0 ± 5.9

6.0 ± 5.5

14.4 ± 9.7

14.1 ± 11.1

10.9 ± 7.8

9.7 ± 14.8

8.4 ± 7.1

5.8 ± 6.5

10.1 ± 10.2

8.3 ± 8.1

6.4 ± 5.0

16.2 ± 20.1

14.8 ± 10.6

11.6 ± 7.5

13.5 ± 20.0

10 9 ± 172

9.1 ± 3.5

14.4 ± 15.2

10.9 ± 9.9

6.8 ± 6.9

15.6 ± 16.6

14.6 ± 13.6

13.6 ± 13.6

14.9 ± 17.5

12.3 ± 13.1

11.4 ± 7.7

14.9 ± 10.5

11.5 ± 10.3

10.3 ± 11.5

26.6 ± 22.8

17.5 ± 13.5

15.8 ± 11.7

27.23 ± 14.3a

10.9 ± 12.9

10.2 ± 15.0

23.9 ± 16.1a

12.4 ± 11.6

11.4 ± 10.7

27.6 ± 23.0

17.8 ± 14.9

14.7 ± 14.8

26.5 ± 26.1a

12.4 ± 9.9

10.4 ± 13.2

26.2 ± 10.6a

12.6 ± 13.4

11.6 ± 10.5

270 mA

Mean ± SD (%)

200 mA

Mean ± SD (%)

120 mA

Mean ± SD (%)

80 mA

Mean ± SD (%)

40 mA

Mean ± SD (%)

20 mA

Mean ± SD (%)

10 mA

Mean ± SD (%)

GGN(-800 HU)

Reconstruction method

FBP

270 mA

Mean ± SD (HU)

200 mA

Mean ± SD (HU)

120 mA

Mean ± SD (HU)

80 mA

Mean ± SD (HU)

40 mA

Mean ± SD (HU)

20 mA

Mean ± SD (HU)

10 mA

Mean ± SD (HU)

PSN (-630 HU)

AIDR 3D

FIRST

58.8 ± 19.7

41.2 ± 8.4*

24.6 ± 12.3*,**

69.7 ± 19.2

45.7 ± 8.3*

23.3 ± 10.21 *,**

87.5 ± 20.9

53.4 ± 11.0*

106.9 ± 25.4

FBP

SN (100 HU) FIRST

FBP

AIDR 3D

61.8 ± 19.3

38.8 ± 17.0*

22.4 ± 7.8*,**

92.2 ± 23.1

73.5 ± 15.3*

30.4 ± 16.2*,**

77.0 ± 23.0

48.2 ± 14.6*

23.3 ± 8.5*,**

105.1 ± 22.7

76.0 ± 13.7*

42.5 ± 24.3*,**

29.3 ± 14.8 *,**

102.1 ± 27.8

55.2 ± 27.8*

28.7 ± 12.7 *,**

138.0 ± 31.0

84.6 ± 12.3*

35.7 ± 17.6*,**

62.9 ± 17.4*

29.5 ± 12.5*,**

117.3 ± 44.3

68.9 ± 18.4*

31.0 ± 31.3*,**

158.1 ± 42.3

91.3 ± 11.0*

37.9 ± 16.8*,**

150.4 ± 36.7

79.1 ± 36.3*

33.5 ± 10.4*,**

176.6 ± 54.8

84.8 ± 17.0*

45.3 ± 31.6*,**

224.4 ± 64.0

103.8 ± 11.9*

51.9 ± 15.0*,**

218.1 ± 52.3

80.1 ± 13.7*

38.8 ± 16.7*,**

251.5 ± 83.7

82.6 ± 7.7*

40.8 ± 8.0*,**

339.1 ± 106.3

109.7 ± 11.7*

54.3 ± 22.7*,**

341.2 ± 70.7

84.8 ± 18.5*

43.5 ± 13.7*,**

380.5 ± 170.8

87.9 ± 12.7*

45.0 ± 12.0*,**

519.4 ± 159.5

127.5 ± 36.9*

75.4 ± 27.0*,**

a

*Significant difference with FBP at the same tube current (p \ 0.05) **Significant difference with AIDR 3D at the same tube current (p \ 0.05)

Results of a comparison of the percentage of absolute measurement errors among the three methods for each nodule type group are shown in Table 2, indicating that the mean percentages of absolute measurement errors of AIDR 3D and FIRST for each nodule type group were significantly lower than that of FBP at 20 mA and 10 mA (p \ 0.05). In addition, percentages of absolute measurement errors of FBP at 20 mA and 10 mA were significantly higher than those at 270 mA for all nodule type groups (p \ 0.05). A comparison of the percentage of absolute measurement errors of AIDR 3D and FIRST at each tube current with that of FBP at 270 mA showed no significant differences (p [ 0.05).

AIDR 3D

SN (100 HU) FIRST

FBP

AIDR 3D

FIRST

Significant difference with absolute error with CT data at 270 mA on each reconstruction method (p \ 005)

Conclusion Our CADv software is tolerant to low-dose CT with all reconstruction algorithms. Moreover, FIRST and AIDR 3D algorithms can maintain measurement accuracy on ultra-low dose CT in chest phantom study. References [1]

[2]

FIRST

SD: Standard deviation

FBP

* Significant difference with absolute error of FBP (p \ 0.05)

[3]

AIDR 3D

PSN (-630 HU) FIRST

SD Standard deviation

Comparison of image noise for the three methods and each nodule type group

Phantom type (HU)

Comparison of the percentage of absolute measurement errors among the three methods for each nodule type group

Phantom type (HU)

[4]

Li Q, Gavrielides MA, Sahiner B, et al. (2015) Statistical analysis of lung nodule volume measurements with CT in a large-scale phantom study. Med Phys.42(7): 3932. Ohno Y, Takenaka D, et al. (2012) Adaptive iterative dose reduction using 3D processing for reduced- and low-dose pulmonary CT: comparison with standard-dose CT for image noise reduction and radiological findings. AJR Am J Roentgenol. 199(4): W477–485. Katsura M, Matsuda I, et al. (2013) Model-based iterative reconstruction technique for ultralow-dose chest CT: comparison of pulmonary nodule detectability with the adaptive statistical iterative reconstruction technique. Invest Radiol. 48(4): 206–212. Nishio M, Matsumoto S, et al. (2014) Emphysema quantification by combining percentage and size distribution of low-attenuation lung regions. AJR Am J Roentgenol. 202(5): W453–458.

A new semi-automated segmentation method for calculating cerebral hematoma volume in brain CT images O. Dandin1, F. Cuce2, U. Teomete3, T. Ergin4, O. Osman5, G. Tulum6 1 Bursa Military Hospital, Department of General Surgery, Bursa, Turkey 2 Ankara Mevki Military Hospital, Department of Radiology, Ankara, Turkey 3 University of Miami Miller School of Medicine, Department of Radiology, Miami, FL, USA 4 Gulhane Military Medical Academy, School of Medicine, Department of Radiology, Ankara, Turkey 5 Istanbul Arel University, Department of Electrical and Electronics Engineering, Istanbul, Turkey 6 Yeni Yuzyil University, Department of Electrical and Electronics Engineering, Istanbul, Turkey Keywords Brain  Hematoma  Segmentation  Tomography Purpose The rate of intracranial hemorrhage in all stroke cases varies between 10 % and 15 %. Additionally, the outcome of the patients with

123

Int J CARS intracranial hemorrhage (ICH) is poorer comparing to ischemic stroke cases [1]. Gold standard diagnostic tool for ICH is computed tomography (CT). Along with neurological status and age of patients, the findings acquired from CT such as the volume of ICH, hydrocephalus, infratentorial location and midline shift can be used for predicting the prognosis of the patients [1,2]. ABC/2 isthe the simple ellipsoid shaped volume calculation formula, which was found by Kwak et al. [3]; obtained by multiplying the three longest hematoma dimensions and dividing the result to 2. This is the most common method utilized for calculating the intracranial hematoma volume, quantitatively. However, this method can lead mistakes in calculations due to the irreguler shape of hematomas. This challenge may affect the management of these patients. Computer Aided Diagnosis (CAD) is a novel image processing technology which has been improved for accurate and fast assessment of patients’ diagnostic data. Also, CAD has been used for automated and semi-automated segmentation of intracranial hematoma and volume measurements [4]. Our aim is to demonstrate a new semi-automated method for measuring intracranial hematoma volume and to introduce a new formula, which provides better accuracy than the conventional method, for computing this volume using dimensions of hematoma. Methods The performance of the proposed methods was evaluated on randomly selected 10 CT scans from 10 patients (5 female, 5 male, mean age 62 years old) with acute subdural and parenchymal hematoma. Four patients had subdural, six had parenchymal hematomas (Fig. 1). A new semi-automated segmentation tool ManSeg 2.4 was used by a radiologist for measuring hematoma volumes and the results were compared with the conventional method (ABC/2 formula) by another blinded radiologist. Midline shift and thickness of the hematoma were also measured (Table 1). Statistical analyzes were performed by using Student’s t test. P \ 0.05 denoted statistical significance. An experimental phantom model, which used a sheep’s solid organs (liver, kidney, spleen and heart) and muscles, was prepared for validation of our segmentation method and volume computation of the semi-automated tool. The results of this study were satisfactory with 98.29 % ± 0.88 and 97.33 % ± 1.77 accuracy rates for 2 mm and 5 mm thicknesses, respectively. In addition, a new volume calculation formula was obtained and compared to the volumes of radiologists’ segmented hematoma and conventional method, ABC/2.

Intracranial hematoma types, diameters, volumes calculated with ABC/2 and ManSeg methods and lengths of midline shift of the patients

Table 1

Volume calculated with ManSeg (ml)

Midline shift (mm)

Outcome

No.

Hematoma Type

A (mm)

B (mm)

C (mm)

Volume calculated with ABC/2 (ml)

1

Intraparenchymal

56.2

57.6

36.9

59.72

54.98

4.9

Excitus

2

Intraparenchymal

44.9

72.1

52.2

84.5

78.27

9.9

Excitus

3

Intraparenchymal

66.2

52.8

54

94.4

186.75

5.9

Alive

4

Intraparenchymal*

Big: 55.2 Small: 20

Big: 59.7 Small: 32

Big: 42.6 Small:15.8

Big: 70,2 Small: 5.1 Total: 75.3

69.34

12.3

Alive

5

Intraparenchymal

11.2

12

9,5

0,6

0.6

None

Excitus

6

Intraparenchymal

37.9

11.4

22.5

4.9

3.46

None

Alive

7

Subdural*

Right:52 Left: 49.7

Right: 122.4 Left: 110

Right: 12.7 Left: 7.4

Right: 40.4 Left: 20.2 Total:60.6

54.43

None**

Alive

8

Subdural

86

138,2

17

101

69.01

12

Excitus

9

Subdural

86

124.2

15.8

84.4

162.04

4.9

Excitus

10

Subdural

55

82.2

5.9

13.3

20.79

3.9

Alive

* Existing two hematoma wit different sizes, **due to bilateral subdural hematoma, A: craniocaudal diameter, B: anteroposterior diameter, C: transverse diameter (hematoma thickness)

For segmenting hematoma region and calculating the hematoma volume manually, we developed ManSeg 2.4 which is a new semiautomated segmentation program. Three different segmentation approaches were included in this software. Each one was used in different situations. The first approached is a manual segmentation and was used only when other methods couldn’t be used. The second method is 2D semi-automated segmentation method and the third one is 3D automated segmentation. In 2D semi-automated segmentation we used active contour method with Chan and Vese’s region based energy model. Our 3D segmentation method works based on the second step of the algorithm given in [5]. Results Measurement of hematoma volume was feasible in all cases. It is inferred that conventional ABC/2 method in volume estimation results in a high error. Therefore, the best way to find the volume of hematoma is to segment the hematoma in CT images. As an alternative way to ABC/2 method, to measure the volume, trilinear approximations were introduced by using length, width and thickness of the hematoma on axial CT images. Least squares technique was utilized for acquiring the formula’s coefficients. In our proposed formula, the hematoma volumes were acquired utilizing the seven terms of the trilinear equation given below. V ¼ a0 W þ a1 T þ a2 L þ a3 WT þ a4 WL þ a5 TL þ a6 WTL We found a strong correlation between hematoma volume calculated by ManSeg 2.4 and clinical outcome rather than midline shift and hematoma thickness. Root mean square error (RMSE) and standard deviation (SD) error of ABC/2 method were calculated as 39.66 ml and 34.22 ml, respectively (Fig. 2). RMSE and SD error of the trilinear method were calculated as 34.15 ml and 30.42 ml respectively.

Fig. 1 Segmentation of intraparenchymal and subdural hematoma

123

Int J CARS

Fig. 2 RMSE (root mean square error) of ABC/2 method Conclusion The gold standard for ICH volume calculation considered to be the segmentation of lesions on CT images. The brain hematoma volume calculation with semi-automated segmentation tool ManSeg 2.4 presents us a reliable option compared to conventional method. This method helps to achieve the best management of the patients with the rapid and accurate total intracranial hematoma volume and will lead automatic methods for detecting most accurate intracranial hematoma volumes. References [1] Rosa Junior M, da Rocha AJ, Maia Junior AC, Saade N, Gagliardi RJ (2015) The active extravasation of contrast (spot sign) depicted on multidetector computed tomography angiography might predict structural vascular etiology and mortality in secondary intracranial hemorrhage. Journal of computer assisted tomography 39 (2):217–221. doi:10.1097/rct.0000000000000182 [2] Wong GK, Tang BY, Yeung JH, Collins G, Rainer T, Ng SC, Poon WS (2009) Traumatic intracerebral haemorrhage: is the CT pattern related to outcome? Br J Neurosurg 23 (6):601–605. doi:10.3109/02688690902948184 [3] Kwak R, Kadoya S, Suzuki T (1983) Factors affecting the prognosis in thalamic hemorrhage. Stroke 14 (4):493–500 [4] Liu B, Yuan Q, Liu Z, Li X, Yin X (2008) Automatic segmentation of intracranial hematoma and volume measurement. Conference proceedings : Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Annual Conference 2008:1214–1217. doi:10.1109/iembs.2008.4649381 [5] Dandin O, Teomete U, Osman O, Tulum G, Ergin T, Sabuncuoglu MZ (2015) Automated segmentation of the injured spleen. International journal of computer assisted radiology and surgery 1–18.

DAR4DCT generator: software to generate augmented reality based 4DCT in the lack of 4DCT image acquisition devices S. Nabavi1, S. Bayat2, M. Mohammadi3, M. Ebrahimi Moghaddam1 1 Shahid Beheshti University, Computer Science and Engineering, Tehran, Iran, Islamic Republic of 2 Islamic Azad University, Computer Science, Hamadan, Iran, Islamic Republic Of 3 Royal Adelaide Hospital, Medical Physics, Adelaide, Australia

therefore, healthy tissues or organ at risks (OARs) may be irradiated [1]. Different methods have been proposed to protect OARs such as tumor motion prediction methods, deep respiration breath hold, respiratory gating, tracking techniques and etc. All of these methods have some advantages and disadvantages but for example, because of high mortality rate in lung cancer cases, new therapy methods should be seriously considered. Four-dimensional RT (4DRT) based on images acquired using 4DCT imaging devices is a modern and new technique that helps cancer treatment team to monitor lung tumor motions to achieve better results in treatment procedure. RT based on 4DCT images was started from 2002 to 2003 and at the moment, number of RT centers that use this technology is rising every day [2]. Although application of this technology should grow, unfortunately many RT centers especially in developing countries do not have access to this imaging method because of economical or technological issues. To fill this gap, we try to develop software that enables RT centers to have 4DCT images in the lack of 4DCT imaging tools. The software provides a considerably visual and mutable environment with board range of image processing facilities aimed at conveying better view of lung moving tumors from conventional CT images which are often devoid of required information of tumor movements. It prepares suitable information about lung tumor motion for radiotherapists to delineate more accurate contours and achieve better results in lung cancer treatment. Methods DAR4DCT generator works based on combination of conventional CT images, tumor motion modeling and augmented reality (AR) concept. This software receives conventional CT images as input and after determination of tumor motion extent and augmentation of this extent to input images based on AR concept, DAR4DCT images are generated as main output. At the first step to generate expected output, canny edge detection algorithm is used to detect objects including tumor tissues in lungs [3]. Next, one of existent models to predict lung tumor motions is used to estimate the extent of tumor motion in patients’ lungs. This springdashpot system based on Voigt model estimates the correlation between lung tumor and abdominal respiratory motions using an external surrogate signal [4]. The average error of mentioned model is about 2.6 mm. Estimated extent of tumor motion changes the detected edges to make a new slice which shows one of respiration phases. Changed detected edges are then augmented to original CT images have been acquired in end-exhale phase of respiration using image fusion techniques. The DAR4DCT images are generated by repeating this process for different slices and phases of respiration. All of aforementioned processes are done using the software developed using Visual studio 2010 and based on rational unified process (RUP) framework. Results The original and generated CT images provided using DAR4DCT generator have been shown in Fig. 1. The left image is related to generated slice of end-inhale phase of respiration in coronal view and the red contour clearly shows the amount of tumor motion due to respiration because the background image is related to end-exhale phase of respiration. The right image shows the original corresponding slice.

Keywords Augmented reality  Four-dimensional CT  Lung cancer  Tumor motion modelling Purpose Moving tumors always are a key challenge in radiation therapy (RT) because the motions continuously change the tumor location;

Fig. 1 Comparison of a DAR4DCT (left) and an original corresponding 4DCT (right) slice related to end-inhale phase of respiration

123

Int J CARS Evaluation of differences between DAR4DCT and original 4DCT edge maps in different phase of respiration from end-exhale to endinhale shows that the error rate of output images toward original corresponding ones is 2.7 ± 0.5 mm in the average. Conclusion The implemented software can be a suitable alternative in the lack of 4DCT image acquisition devices. Less error rate, reduction of imaging dose delivered to patients and time consuming post-processing actions to eliminate artifacts toward original 4DCT images acquired using 4DCT imaging devices, highly interactive and userfriendly interfaces, strong medical image processing toolbox with diverse range of functions and manual contouring application are the key features of the software. Based on information provided, the software can significantly compensate the lack of 4DCT imaging technology. Results also clearly show that augmented reality can promote the viewer’s perception of generated images. References [1] Khan F, Bell G, Antony J, Palmer M, Balter P, Bucci K, et al. ‘‘The use of 4DCT to reduce lung dose: A dosimetric analysis,’’ Medical Dosimetry, vol. 34, pp. 273–278, 2010. [2] Ehrhardt J, Lorenz C 4D modeling and estimation of respiratory motion for radiation therapy: Springer, 2013. [3] Mohammadi M, Nabavi S ‘‘Standard edge detection algorithms versus conventional auto-contouring used for a three-dimensional rigid CT–CT matching,’’ Iranian Journal of Radiation Research, vol. 10, pp. 123–130, 2012. [4] Ackerley E, Cavan A, Wilson P, Berbeco R, Meyer J ‘‘Application of a spring-dashpot system to clinical lung tumor motion data,’’ Medical physics, vol. 40, p. 021713, 2013.

Quantification of bone microarchitecture in ultra-high resolution extremities cone-beam CT with a CMOS detector and compensation of patient motion E. Marinetto1,2, M. Brehler1,3, A. Sisniega1, Q. Cao1, J. W. Stayman1, J. Yorkston4, J. Siewerdsen1,5, W. Zbijewski1 1 Johns Hopkins Univ, Biomedical Engineering, Baltimore, United States 2 University Carlos III de Madrid, Madrid, Spain 3 German Cancer Research Center, Heidelberg, Germany 4 Carestream Health, Rochester, NY, United States 5 Johns Hopkins Univ, Radiology, Baltimore, United States Keywords Cone-beam CT  CMOS X-ray detector  Extremities imaging  Bone microarchitecture Purpose Quantitative metrics of bone microarchitecture are a biomarker for applications ranging from early detection of osteoarthritis to assessment of fracture risk in osteoporosis. Bone morphometry has been limited, however, to pre-clinical studies with micro-CT (lCT). Its widespread diagnostic adoption is hampered by the lack of a clinical modality capable of sufficiently high spatial resolution to enable accurate measurements of trabecular and cortical microarchitecture in patients. Currently, clinical assessment of bone morphology is limited to distal extremities using specialized high resolution peripheral quantitative CT systems. Preliminary studies of extremities conebeam CT (CBCT) using flat-panel detectors (FPDs) [1] indicate that such scanners could provide a platform for in vivo bone morphometry owing to the high resolution of the FPD [2], while maintaining soft tissue visibility comparable to conventional CT and providing large field of view and weight-bearing imaging. We report on the next generation extremities CBCT based on a CMOS X-ray detector offering smaller pixel size and reduced electronic noise compared to FPDs. We investigate the accuracy of

123

metrics of bone architecture obtained on the CMOS-based system in comparison to gold standard lCT and FPD-based CBCT. In addition to the optimized detector, achieving the resolution required for bone morphometry in clinical scenarios requires compensation of involuntary sub-mm motions that cannot be completely mitigated by patient immobilization. We propose a fiducial-free motion compensation based on an image sharpness criterion. Methods Ultra-high resolution CMOS-based extremities CBCT: The extremities CBCT allows imaging of weight-bearing lower extremities in a natural stance and non-weight bearing upper and lower extremities. The scan time is *20 s for a 20 x 20 x 20 cm3 field of view, the patient dose is 6–10 mGy. The current prototype uses an FPD with 139 lm pixel pitch (PaxScan2530, Varian) and a stationary anode X-ray source with 0.5 focal spot (XRS-125–10 K-P, SourceRay). The system is being upgraded to an ultra-high resolution CMOS detector (Dalsa Xineos 3030) with 99 lm pixel pitch and a compact rotating anode X-ray source with 0.3 focal spot and 3 kW power output (IMD RTM 37). Quantitative assessment of bone microarchitecture: A cadaveric wrist was imaged on a testbench in the extremities CBCT configuration using the CMOS and FPD detectors (in situ scans) and reconstructed using the Feldkamp algorithm with 75 lm voxels. Wrist bones were excised and scanned ex situ on a lCT system (SkyScan 1172, voxel size 14 lm). For each bone, rigid registration was applied to align the lCT, CMOS-CBCT and FPD-CBCT volumes. In the distal ulna, 23 cubic ROIs (4 mm3) were randomly placed inside the bone. Connectivity-maximizing thresholding was applied to segment the trabecular matrix. Conventional metrics of trabecular microarchitecture (e.g. Trabecular Spacing Tb.Sp, Bone Volume Fraction BV/TV) were computed. Image-based motion compensation: We use an iterative approach which seeks a motion trajectory that maximizes an objective function consisting of a gradient metric encouraging image sharpness and a penalty term encouraging smooth motion. This ‘‘auto-focus’’ method [3,4] assumes that higher values of the objective function correspond to reduced motion-induced artifacts in the image. The reconstructions are computed using the Feldkamp algorithm (75 lm voxels) by applying the current guess for motion trajectory during the backprojection. The Covariance Matrix Adaptation Evolution Strategy is used in the maximization. One of the advantages of this approach is that the estimation of the motion can be limited to a local ROI and thus a locally rigid motion model can be used. The method was evaluated on a CMOS-CBCT testbench emulating the extremities system. An anthropomorphic wrist phantom was scanned (720 projections, 90 kVp). The motion patterns applied to the phantom involved a smooth step function of 5–20 mm amplitude in the (x,y) plane moving at a rate of 1 mm/1o. Results Figure 1 compares ROI images used in assessment of bone morphometry. The improved spatial resolution of CMOS-CBCT compared to the current FPD-CBCT is evidenced by better visualization of the trabeculae. CMOS-CBCT showed improved agreement with lCT compared to FPD CBCT for both Tb.Sp and BV/TV. Median BV/TV was 0.18, 0.33 and 0.41 for lCT, CMOS CBCT and FPD CBCT, respectively. Tb.Sp was 0.66 for lCT, 0.7 for CMOSCBCT and 0.8 for FPD-CBCT. While there is some deviation in the numerical value of the metrics of bone microarchitecture between lCT and CMOS-CBCT, the clinical application of such metrics is in detection of pathological change in bone morphology. This type of relative measurement can be reliably performed as long as there is a correlation between the structural metrics obtained on CBCT and lCT, even if there is no absolute numerical agreement. Ongoing work investigates the correlation between the metrics obtained on CMOSCBCT and lCT using a set of osteoporotic biopsy cores.

Int J CARS

Fig. 1 Micro-CT (left), CMOS-based CBCT (center), and FPD-based CBCT (right) images of one of the ROIs used for analysis of the metrics of bone microarchitecture Figure 2 illustrates the performance of the image-based motion compensation in reduction of artifacts and improvement in visualization of the trabeculae and delineation of bone boundary (arrows). The Root Mean Squared Error (RMSE) with respect to the static image was reduced more than two-fold after motion compensation for the case of 20 mm step motion shown here.

Conclusion The implementation of an ultra-high resolution CMOS X-ray detector in extremities CBCT yielded increased spatial resolution that resulted in improved accuracy of bone morphometry. The image-based, fiducial-free motion compensation restored visualization of the trabecular matrix for a wide range of motion amplitudes. The CMOS-based extremities CBCT will combine ultra-high spatial resolution, soft-tissue visualization and ability to image under load to yield a platform for comprehensive quantitative assessment of the musculoskeletal system. Research support: EU FP7 IRSES TAHITI (#269300), FEDER funds and NIH R01-EB-018896. References [1]

[2]

[3]

[4]

Carrino JA, Siewerdsen JH, Muhit AA, Zbijewski W, Stayman JW, Yorkston J, Packard N, Yang D, Senn R, Foos D, Thawait G (2014) Dedicated Cone-Beam CT System for Extremity Imaging. Radiology 270(3):816–824. Muhit AA, Arora S, Ogawa M, Ding Y, Zbijewski W, Stayman JW, Thawait G, Packard N, Senn R, Yang D, Yorkston J, Bingham CO, Means K, Carrino JA, Siewerdsen JH (2013) Peripheral quantitative CT (pQCT) using a dedicated extremity cone-beam CT scanner. Proc. SPIE 8672:8672031-7. Kingston A, Sakellariou A, Varslot T, Myers G, Sheppard A (2011) Reliable automatic alignment of tomographic projection data by passive auto-focus. Medical Physics 38(9):4934. Sisniega A, Stayman JW, Cao Q, Yorkston J, Siewerdsen JH, Zbijewski W (2016) Image-based motion compensation for highresolution extremities cone-beam CT. Proc. SPIE 9783: 97830K1-7.

Semi-automated quantitative vertebral morphometry: CT scout-based digital image enhancement reliability study J. Narloch1,2,3, W. Glinkowski1,2 1 Medical University of Warsaw, Medical Informatics and Telemedicine, Warszawa, Poland 2 Baby Jesus Clinical Hospital, Orthopaedics and Traumatology of Locomotor System, Warszawa, Poland 3 Medical University of Warsaw, Chair and Department of Orthopaedics and Traumatology of Locomotor System, Warszawa, Poland Keywords Semiautomated morphometry  Vertebral fractures  CT scouts  Digital image enhancement

Fig. 2 CBCT image obtained without motion (top) is compared with image contaminated by a 2 cm step motion (center) and a reconstruction of the motion-contaminated data processed with the motion compensation (bottom)

Purpose Surgery of vertebral fractures depends on accurate radiographic diagnosis. Radiographic evaluation of osteoporotic vertebral fracture is of utmost importance since they occur in around 20 % of postmenopausal women, 75 % elude clinical attention, and qualified readers do not diagnose 30 %. Undetected fractures remain untreated, what accounts for high morbidity. Lateral radiograph of the spine is currently considered a gold standard for vertebral fracture diagnosis. International Society for Clinical Densitometry recommends dual-energy X-ray absorptiometry for vertebral fracture diagnosis [1]. CT scout or scanogram is a 2D digital radiograph intended for localization of the area of interest prior to the actual scan. Although its resolution is inferior to a conventional radiograph, it could be a useful modality especially with the increasing number of thoracoabdominal CT examinations performed for various indications—Fig. 1.

123

Int J CARS

Fig. 2 Initial CT scanogram (no digital enhancement) Fig. 1 An example of scanogram with obscure bone structure. Note the presence of multiple vertebral fractures There are three techniques to identify vertebral fractures from lateral spine views: Genant’s semiquantitative visual scale, algorithm-based qualitative approach and quantitative six-point morphometry. Their limitations are high level of expertise and high point placement variability [2–4]. Software assistance substantially reduces the time commitment (average of 5 min for our program vs. 15 min for manual six-point placement) and necessary reader’s training. Methods We followed two paths in our investigation. Firstly, we ventured to establish if the reliability between raters improves after image enhancement. We investigated intra-reader agreement similarly. For this purpose, we compared analyses, which were reviewed, and their point placements were adjusted manually if necessary, with those with no reader intervention. Mentioned analysis was performed on two sets of pictures: first with original CT scouts and second, where scouts underwent digital modification (to improve edge detection). Secondly, we tried to verify if different methods of image modification would change (hopefully improve) fracture detection if there were no manual adjustment of point placement. That would reveal if digital enhancement facilitates segmentation by the given software and in turn detect more fractures. 250 computed tomography (CT) scanograms of patients suffering from osteoporosis and with or without prevalent vertebral fractures, admitted to Orthopaedic Trauma Department were analyzed—Fig. 2.

123

Semi-automated quantitative vertebral morphometry was performed using a model-based shape recognition algorithm. It uses standard six-point morphometry enriched with detailed 95-point landmark annotation to capture the shape of each vertebra (SpineAnalyzer, Optasia Medical, Cheadle, UK). DICOM-derived lossless Tiff images were loaded. The reader labeled each vertebral body manually with a single point located in the approximate center of each vertebra. These serve as a landmark for the algorithm to identify vertebral body contours (95-points) and to place six points for the standard morphometry. The reader reviews the annotation, and if necessary, manually changes the point placement—each of the 95 points can be easily adapted. The program computes vertebral, height ratios, and deformity percentage indicative of vertebral fracture based on six-point morphometry. We used ImageJ 1.46r (Wayne Rasband, NIH, USA), an opensource Java-based image processing application, to perform digital image enhancement. The original set of images was triplicated and digitally modified using different techniques. The first set was SHARPENED, the second set underwent UNSHARP MASKING, and the remaining set was CONVOLUTED using 9 x 9 kernels. For the purpose of the study, manually adjusted morphometry was considered a standard while four automated analyzes were evaluated for accuracy in discovering vertebral fractures. Two non-radiologists performed the morphometric analysis on the original set of images to determine Intra- and inter-reader reliability. Measurements were repeated [ 3 weeks later. Results The morphometricanalysis produced 1485 vertebrae, 200 of which were classified as fractured, according to Genant’s, where deformity percentage C 20 % is considered a fracture.

Int J CARS Unadjusted morphometry found 63 fractures, 33 of which were true positive (AUTOMATED with no digital enhancement), SHARPENING detected 57, with 30 true positive. UNSHARP MASKING yielded 30, with 13 true positive. In the CONVOLUTION set 24 were discovered, with nine true positive. Diagnostic odds ratio was C 1 in all cases. Values reached 8.27 (95 % CI 4.91–13.91, Z = 7.959, p Binomial proportions for all techniques were computed, where 95 % confidence intervals for sensitivity were 0.113–0.216, observed ASE 0.026; 0.100–0.199, observed ASE 0.025; 0.031–0.099, observed ASE 0.017; and 0.016–0.074, observed ASE 0.015 for AUTOMATED, SHERPENING, UNSHARP MASKING and CONVOLUTION, respectively. Considering all vertebral levels, intra-rater reliability for height ratios, we found kappa ranging from 0.22 to 0.41 for adjusted measurements and 0.16 to 0.38 for unadjusted. Agreement for prevalent fractures, based on Kappa ranged from 0.29 to 0.56, and from -0.01 to 0.23 for adjusted and unadjusted measurements respectively. Conclusion We evaluated the capacity of one of the existing to detect fractures semi-automatically, with the assistance of digital image enhancement. The results suggest the unadjusted morphometry is not yet feasible and accurate enough to achieve its place in a clinical setting. Acknowledgement: This study was supported by grant N404 695940 financed by National Science Center and by the project NR13-0109-10/2010 funded by the National Centre for Research and Development References [1] Delmas PD, van de Langerijt L, Watts NB, Eastell R, Genant H, Grauer A, Cahall DL, I.S. Group Underdiagnosis of vertebral fractures is a worldwide problem: the IMPACT study. J Bone Miner Res, 2005. 20(4): p. 557–63. [2] Genant HK, Wu CY, van Kuijk C, Nevitt MC Vertebral fracture assessment using a semiquantitative technique. J Bone Miner Res, 1993. 8(9): p. 1137–48. [3] Jiang G, Eastell R, Barrington NA, Ferrar L Comparison of methods for the visual identification of prevalent vertebral fracture in osteoporosis. Osteoporos Int, 2004. 15(11): p. 887–96. [4] Hurxthal LM Measurement of anterior vertebral compressions and biconcave vertebrae. Am J Roentgenol Radium Ther Nucl Med, 1968. 103(3): p. 635–44.

Synergism of ultrasound and magnetic resonance imaging in interventional radiology A. Melzer1,2, (for the TransFusimo FP7 Consortium) ICCAS, University Leipzig, Germany 2 IMSaT, University Dundee, United Kingdom 1

Keywords Ultrasound  Magnetic Resonance Imaging  Focused ultrasound  MRI guided Focused Ultrasound Purpose Ultrasound and Magnetic Resonance Imaging MRI can be used sequentially to control conventional needle and catheter based interventions. The ‘‘in room’’ use of ultrasound in the MRI requires certain technical developments [1]. MR guided Focused Ultrasound MRgFUS is an established non-invasive treatment method for stable solid tumors. MRgFUS treatment of liver is an alternative to conventional surgery and RF ablation but has two challenges: the presence of the ribcage, and the motion of liver due to respiration. The bone absorbs energy and gets heated which is causing pain and necrosis. Respiratory motion during FUS would lead to ablate healthy

tissue, vasculature and can therefore cause significant complications. The procedure of MRgFUS of liver can currently only be performed under apnoea and only to reach caudal lesions in the liver lobes [2]. Our current research includes robotic positioning of FUS (Fig. 1) and interactive control of both MRI and FUS to solve these problems.

Fig. 1 Schematic view of MR guided Robotic position of Diagnostic and Focused Ultrasound Methods To overcome these challenges, novel controller was designed (TransFusimo) to control MRI (GE 1.5 T Milwaukee, USA) and sonicate with multiple selective-element ultrasonic transducer (InSightec, Israel). Protocols are designed to check treatment parameters for quality assurance. Control parameters that need to be monitored are sonication duration, location, power and temperature for successful application of MRgFUS. Simultaneous use of diagnostic Ultrasound inside an MRI has been accomplished by shielding of conventional Ultrasound with elongated cables (10 m) with aluminium foils and copper mesh but more suitable is wireless transmission of Ultrasound probe acquired data to the image processing and display unit. In addition the positioning of the ultrasound and therapeutic probes have be achieved by a modified MRI compatible robotic system, Innomotion, IBSMM, CZ) [3]. The setup for both solutions has been validated on ex vivo phantom models and ex vivo human cadavers. Results Feasibility tests show that Ultrasound probe positioning can be achieved within a precision of ±2 mm and +7 to 2. The newly developed MRI Focused Ultrasound control protocols can check input and output successfully for control of MRgFUS application to achieve a thermal lesion under simulated respiratory motion (20 mm 16 times/ min). Conclusion In room use of diagnostic and focused Ultrasound inside the MRI is technical feasible and can be used to complement the two imaging modalities both for image guided diagnostic and therapy. References [1] Petrusca L, Viallon M, Terraz S, de Luca V, Celicanin Z, Auboiroux V, Brunke S, Cattin P, Salomir R (2013) Simultaneous Ultrasound Imaging and MRI Acquisition. Interventional Magnetic Resonance Imaging Part of the series Medical Radiology pp 457–470, Springer Berlin Heidelberg. [2] Aubry J-F, Butts Pauly K, Moonen Ch, ter Haar G, Ries M, Salomir R, Sokka S, Sekins MK, Shapira Y, Ye F, Huff-Simonin H, Eames M, Hananel A, Kassell N, Napoli A, Ha Hwang J, Wu F, Zhang L, Melzer A, Kim, Y-s, Gedroyc WM (2013) The road to clinical use of high-intensity focused ultrasound for liver cancer: technical and clinical consensus, J Ther Ultrasound. 1:(13) 1–13. [3] Melzer A, Gutmann B, Remmele T, Wolf R, Lukoscheck A, Bock M, Bardenheuer H, and Fischer H (2008) INNOMOTION

123

Int J CARS for percutaneous image-guided interventions. IEEE Eng. Med. Biol. Magazine 27 (3) 66–73.

Evaluation of tongue squamous cell carcinoma resection margins using ex vivo MR S. Steens1, E. Bekers2, W. Weijs3, G. Litjens2, A. Veltien1, A. Maat2, G. van den Broek4, J. van der Laak2, J. Fu¨tterer1, C. Hulsbergen - van de Kaa2, T. Merkx3, R. Takes4 1 Radboudumc, Radiology and Nuclear Medicine, Nijmegen, Netherlands 2 Radboudumc, Pathology, Nijmegen, Netherlands 3 Radboudumc, Oral and Maxillofacial Surgery, Nijmegen, Netherlands 4 Radboudumc, Otorhinolaryngology and Head and Neck Surgery, Nijmegen, Netherlands Keywords MR  Oncology  Surgery  Validation Purpose Tongue squamous cell carcinoma (TSCC) is primarily treated surgically. The goal of surgical resection is to remove the tumor with adequate resection margins. An adequate resection margin implies that the tumor is removed in total with a small rim of surrounding normal tissue. If the resection margin is adequate, there is no need for additional treatment such as repeated surgery or (chemo-) radiotherapy, and functional impairment related to the procedure is as low as possible [1–2]. During surgery, information on the resection margin is not available, because macroscopic evaluation of a tumor in the tongue by palpation is difficult and inaccurate, and peroperative biopsies or frozen sections may suffer from sample error. Information on the resection margin during surgery could aid in both obtaining sufficient margins (obviating the need for additional treatment) and minimizing resection volume (limiting functional disability), with better oncologic and functional outcomes, better quality of life and lower costs. In the past, one study evaluated the depth of invasion of TSCC using 1.5T MR ex vivo [3]. To the best of our knowledge however, this is the first study on the feasibility and validity of ex vivo 7T MR to establish the resection margin status in TSCC specimens, using histopathology as gold standard. Methods We performed a non-blinded validation of ex vivo 7T MR to detect the TSCC and resection margin using histopathology as gold standard after medical ethics committee approval and informed consent. Patients received standard preoperative workup, surgery, and postoperative treatment when indicated. Ex-vivo small-bore 7T MR of 10 fresh tongue specimens was performed with turbo-spin echo and diffusion-weighted imaging sequences. After the MR examination, the specimens were formalin-fixed overnight and cut in 3 mm-thick slices, paraffin-embedded, processed and cut in 4 lm thin tissue sections, and stained with haematoxylin and eosin. The maximal invasion depth of the tumor and minimal resection margin for the specimen were evaluated for ex vivo MR and histopathology in a nonblinded fashion. Results In six of seven specimens that showed an invasion depth of the tumor of C 3 mm at histopathology, the tumor could be recognised and delineated on the combination of MR sequences. In one of seven specimens with an invasion depth of the tumor of C 3 mm at histopathology the tumor was visible but delineation on MR was difficult. In the three specimens with an invasion depth of the tumor of \ 1 mm and containing mainly dysplasia with only minor invasive carcinoma components, the tumor was not visible on MR. In the specimens where the tumor could be delineated, resection margin as

123

measured on MR was within a 2 mm range as compared to histopathology. Conclusion We have shown the feasibility and validity of ex vivo 7T MR to establish resection margins in fresh TSCC specimens with an invasion depth of C 3 mm. The additional information from MR on the resection margin may guide peroperative biopsies or frozen sections and/or immediate additional resection during the same surgical procedure. Further studies are needed to prove our concept in a larger patient group, with blinded observers and a more prevalent clinical MR system. References [1] Smits RW, Koljenovic´ S, Hardillo JA et al. Resection margins in oral cancer surgery: Room for improvement. Head Neck 2015 Apr. [Epub ahead of print]. [2] Hinni ML, Ferlito A, Brandwein-Gensler MS et al. Surgical margins in head and neck cancer: a contemporary review. Head Neck 2013;35:1362–70. [3] Tetsumura A, Yoshino N, Amagasa T et al. High-resolution magnetic resonance imaging of squamous cell carcinoma of the tongue: an in vitro study. Dentomaxillofac Radiol 2001;30:14– 21.

Development of a quality assurance phantom and software module to enhance DWI and DWIBS efficiency in diagnosis of benign and malignant tumors K. Sergunova1, I. Karpov2, A. Gromov1, A. Morozov2 Research and Practical Center of Medical Radiology, Health, Moscow, Russian Federation 2 Central Institute of traumatology and orthopedics named after N. N. Priorov, Health, Moscow, Russian Federation 1

Keywords Diffusion-weighted imaging  Tumor  Malignancy  Benign Purpose Although diffusion-weighted imaging (DWI) and diffusion-weighted whole body imaging with background body signal suppression (DWIBS) now plays an important role in tumor diagnosis and the value of apparent diffusion coefficient (ADC) may be useful for distinguishing between malignant and benign tissues [1–3], it can’t be fully applied in differentiating benign from malignant. This can be explained by the fact that MRI-determined ADC values are not controlled during the quality assurance (QA) procedure. Moreover, the signal intensity of neoplastic entities on DWIBS depends on types of post-processing filters and characteristics of RF-coils, and thus can vary from the selected area: head, abdomen, the knee joint, etc. The aim of the study was to develop a DWI/DWIBS QA phantom and software to improve the efficiency and accuracy of tumor diagnosis. Methods Twenty materials generating signal in DWI/DWIBS with different values of viscosity and molecular size were analyzed. According to the experimental results a quality assurance (QA) phantom comprising four vials materials that are capable of simulating a reference signal with a given intensity on DWI and DWIBS was developed. Using these vials we extracted and analyzed data from the medical records of 235 patients C 18 years old presenting with benign and malignant tumors (Fig. 1a), including degenerative-dystrophic and inflammatory changes, obtained at the Central research institute of traumatology and orthopaedics of N.N. Priorov (FGBI CITO). The scans were acquired on a Philips 1.5T Ingenian MRI scanner using whole-body T1-weighted, STIR, DWI/DWIBS imaging with a slice thickness of 5 mm, the number of data acquisition (NEX) equal to 2, b-factor equal to 0, 1, 250, 500, 750, 1000,1500.

Int J CARS Quantitative estimation of IVIM parameters and histogram analysis of diffusion MRI in Ewing’s sarcoma family of tumours

Fig. 1 K. 48Y. Adenocarcinoma of the right kidney. MRI, axial DWIBS tomography using phantom for modeling reference signal intensity during diagnostic scanning (a), segmentation of the detected structures in Matlab, followed by construction of 3D-surface signal intensity distribution function (b) The algorithm for subsequent MR-data processing was implemented in Matlab. It consists of four major components: tumor detection, segmentation, contour alignment and determination of various parameters of the detected structures, such as the center of mass, radius of curvature, the signal intensity ratio of detected structure/phantom, the distribution function of the signal intensity in the region of interest and etc. The algorithm uses analysis of isolablecontour maps (Fig. 1b) and wavelet transform. The user interaction is only needed to define the set of parameters and the region of interest (ROI) before starting automated tracking of tumor on DWI (DWIBS) images and performing calculations. Results We identified two silicon-organic compounds with ADC values of 0.08 and 0.01 mm2/s. Studies have shown high stability of the signal received from these materials over a long period of time (up to 6 months).This allowed to develop phantom which is suitable for modeling the whole range of ADC values and signal intensities on DWIBS images from normal to pathological tissue, including benign and malignant tumors. It can be used both for quality assurance procedures and modeling reference signal intensity during diagnostic scanning. The developed phantom and software module achieved stability of signal intensity using different RF-coils, effectiveness for tumor segmentation and convenience during performing routine screening tests in patients. Conclusion The developed quality assurance phantom and software module can be used to enhance DWI and DWIBS efficiency in diagnosis of tumors. Furthermore, subsequently automatically calculated parameters described above could be used for differentiation benign and malignant tissues. References [1] Awad F, Threshold of Apparent Diffusion Coefficient in the Differentiation between Benign and Malignant Breast Lesions on MR Mammography. Journal of Medical Diagnostic Methods Vol.72 Issue 3 -December,2009- P. 381 -387. [2] Holzapfel K, Duetsch S, Eiber M, Holzapfel K, Rummeny E, Gaa J, Value of diffusion-weighted MR imaging in the differentiation between benign and malignant cervical lymph nodes. European Journal of Radiology-Vol. 4 Issue 2 -May, 2015. [3] Morozov A, Karpov I, Patrickeyev E. Diffusion-weighted magnetic resonance imaging of the lumbar-sacral spine (DWIBS) in patients with bone disease. II congress of the National Society of Neuroradiology: Sat. scientific. tr. Moscow, 2014.—July 4–5.—S. 25–26.

E. Baidya Kayal1, D. Kandasamy2, K. Khare3, J. Tiru Alampally2, S. Bakhshi4, R. Sharma2, A. Mehndiratta1,5 1 Indian Institute of Technology Delhi, Centre for Biomedical Engineering, New Delhi, India 2 All India Institute of Medical Sciences, New Delhi, Department of Radiology, New Delhi, India 3 Indian Institute of Technology Delhi, Department of Physics, New Delhi, India 4 All India Institute of Medical Sciences, BRA IRCH, New Delhi, India 5 All India Institute of Medical Sciences, Department of Biomedical Engineering, New Delhi, India Keywords IVIM  IVIM parametric map  Histogram analysis  Diffusion MRI Purpose Diffusion-weighted imaging (DWI) characterizes the random microscopic motion of molecules and enables assessment of tissue microstructure. The fast attenuation of signal at low b values (0–100 s/mm2) has been shown to capture the perfusion information within the capillary network called Intra-voxel Incoherent Motion (IVIM) [1]. Using quantitative analysis of IVIM effect, both diffusion and perfusion component of tissue can be assessed separately; biexponential model has been widely used for the same [2]. Perfusion information available with IVIM is not very reliable for clinical interpretation. We implemented regularized parametric estimation for IVIM using penalty function Total Variation with bi-exponential model in patients with Ewing sarcoma. Histogram analysis on estimated parametric maps was carried out for extracting more information in tissue characterization. Methods IVIM dataset from four patients (M:F = 3:1, Age = 25.3 ± 8.8 years), with Ewing sarcoma were acquired under the Institutional Review Board approved protocol. The acquisition were performed using 1.5T Philips Achieva MRI scanner with Spin Echo Planar imaging (SP-EPI) sequence with TE = 66 ms,TR = 1782 ms, 5 mm slice thickness and 144x144 matrix size. The DW images were acquired at 11 b-values (0, 10, 20, 30, 40, 50, 80, 100, 200, 400, 800 s/mm2). For one patient DWI images were acquired again after 2 cycles of chemotherapy. Thus in total five IVIM datasets were processed. Apparent diffusion coefficient (ADC) was estimated using mono-exponential model (ME) for b C 200 s/mm2. The IVIM parametric maps for Diffusion coefficient (D), Perfusion coefficient (D*), Perfusion fraction (f) were estimated using BiExponential (BE) model [2] and bi-exponential model with Total Variation (TV) penalty function [3] (BE + TV). An iterative optimization was performed using a nonlinear least square fitting algorithm along with the penalty functions in BE + TV method. Tumour and normal tissue volume were extracted from estimated parametric maps separately using ROIs drawn manually by a radiologist ([5 years of experience). Next, histogram analysis [4] was performed on extracted D, D* and f parameter maps of tumour and normal tissue volume with following quantitative factors: mean, standard deviation, kurtosis and skewness. All analyses were performed by in-house software developed in Matlab R2013b. Results Estimated parametric maps for Diffusion coefficient (D), Perfusion coefficient (D*), Perfusion fraction (f) of the patient with baseline and follow-up data are shown in Fig. 1. Both ADC and D in tumour

123

Int J CARS demonstrated an increase in follow-up images. D* showed no change whereas perfusion fraction (f), demonstrated an increase in perfusion in tumour ROI after chemotherapy. Table 1 depicts the mean values of ADC, D, D* and f for tumour and normal tissue volume for all baseline data. Both ADC and D were lower in tumour than normal tissue, demonstrating a restricted diffusion in tumour as expected [1, 2].

Fig. 1 a,f) DWI image (b = 800 s/mm2); b,g) ADC map; c,h) Diffusion coefficient (D); d,i) Perfusion coeffieicnt (D*); e,j) Perfusion fraction (f) for one representative patient. a-e) baseline; f-j) follow-up after 2 cycles of chemotherapy. Shows increase in perfusion fraction in tumour post chemotherapy Table 1 Mean Apparent Diffusion Coefficient(ADC), Diffusion coefficient (D), Perfusion coefficient (D*) and Perfusion fraction (f) in tumour and normal tissue volume for all baseline data. Mean ADC and D were lower in tumour than normal tissue. Intergroup difference of mean D and D* in normal & tumour tissue observed to increased with BE + TV method compare to BE method Parameters

ADC (910-4 mm2/s) Diffusion (D)

ME Method

BE-Method

Normal

Tumour

7.98 ± 0.82

6.02 ± 0.72

Normal –

BE + TV Method Tumour –

Normal –

Tumour –

–

–

9.79 ± 0.20

9.08 ± 0.57

13.28 ± 2.35

5.5 ± 0.79

–

–

2.95 ± 0.05

2.94 ± 0.04

15.38 ± 2.06

10.75 ± 0.63

–

–

29.58 ± 0.45

29.35 ± 0.45

47.68 ± 3.28

47.73 ± 4.26

-4

(910 mm2/s)

Perfusion (D*) -3

(910 mm2/s)) Perfusion fraction (f) (%)

After applying BE + TV method, intergroup difference of mean D and D* values in normal and tumour tissue were observed to increased than the values evaluated by BE method. In histogram analysis, mean skewness and kurtosis of D in tumour (2.37 ± 0.78 & 12.27 ± 5.7 respectively) were higher than normal tissue (0.59 ± 0.25 & 4.67 ± 1.27 respectively). Mean skewness and kurtosis of D* in tumour (6.8 ± 1.86 & 16.9 ± 2.63 respectively) were also higher than normal tissue (1.83 ± 0.32 & 3.0 ± 0.42 respectively) in baseline. A decrease in kurtosis of D from 13.8 at baseline to 6.8 at follow-up and skewness from 2.4 at baseline to 1.3 at followup were seen for the patient data acquired pre and post chemotherapy. Kurtosis of perfusion (D*) was also reduced from 16.9 at baseline to 14.9 at follow-up after chemotherapy. Conclusion We have used spatial smoothening technique Total Variation penalty function that preserved the physiological information in the generation of IVIM parametric maps that may be helpful in clinical interpretation as compared to standard BE model. Histogram analysis showed useful discriminating features between tumour and normal tissue as well as may help in tumour characterization during baseline & follow-up of Ewing sarcoma. Total Variation optimization with biexponential model and further histogram analysis might be useful in differentiating normal and tumour tissue and response evaluation after chemotherapy in Ewing sarcoma.

123

References [1] Le Bihan D, Breton E, Lallemand D, Aubin ML, Vignaud J, Laval-Jeantet M (1988) Separation Of Diffusion And Perfusion In Intravoxel Incoherent Motion MR Imaging. Radiology Aug; 168(2):497–505. [2] Koh DM, Collins DJ, Orton MR (2011) Intravoxel Incoherent Motion in Body Diffusion-Weighted MRI: Reality and Challenges. AJR June;196:1351–1361. [3] Rudin LI, Osher S, and Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Physica; D(60): 259–268. [4] Mardia KV (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika; 57(3): 519–530.

Collagen ultrastructure determination using angle sensitive MR imaging D. Brujic1, K. Chappell1, Q. Herreros1, J. McGinley1, M. Ristic1 1 Imperial College, Mechanical Engineering, London, Great Britain Keywords Collagen fibre structure  Magic angle  Registration  Segmentation Purpose Knowledge of the orientation of collagen fibers (CF) is essential for diagnosis of injuries as well as planning and developing patient specific treatments. Collagen structures may be determined using multiple scan sets performed in a number of deliberate orientations to magnetic field [1, 2]. To enable in vivo scanning a new MR system is constructed and new scanning procedure is suggested. The main emphasis of this paper is on reducing the time needed for a scanning procedure by minimising the number of orientations and significantly reducing the analysis time, over 100 times faster than in [1, 2]. Methods The novel open MRI system (Fig. 1a) has the main field B0 parallel to the poles, which comprise a profiled arrays of permanent magnets. Rotation about two orthogonal axes achieves all necessary angulations.

Fig. 1 a) The MRI system; b) scanning directions Orientation of the CF is computed in following 4 steps: 1. Fiducial markers are localised in each of N volumes scanned at different orientations, using template matching method implemented in 3D. 2. Voxel correspondences are established by aligning scanned volumes using least squares fitting of corresponding markers [3]. 3. Segmentation is performed using the maximum intensity differences between the corresponding voxels and threshold. Region of interest is interactively selected. Result is shown in Fig. 2c.

Int J CARS [3]

Galassi F, Brujic D, Rea M, Lambert N, Desouza N, Ristic M (2015) Fast and accurate localization of multiple RF markers for tracking in MRI-guided interventions. Magnetic Resonance Materials in Physics Biology and Medicine, 28(1):33–48.

Building 3D statistical shape model of Ulna and Radius bones in MRI Data Sets Fig. 2 a) Ligament appears black when aligned with B0, b) stronger signal when close to the magic angle, c) segmented ligament d) all collagen rich voxels, e) computed CF orientation 4. For each voxel of segmented volume, Fig. 2d, collagen orientation was determined [1]. Orientation of collagen fibre is computed for the collagen rich voxels only. This makes the analysis more than 100 times faster than in [1, 2]. The result is visualised in Fig. 2e. The key achievement presented in this paper is the minimisation of the number of scans realised through determination of optimal scanning directions. Previous work [1, 2] suggested 15 and 35 ad hoc orientations while we have achieved significantly better accuracy with only 9 directions. These directions were determined by finding a set of N points that minimises the maximum distance between any point on a hemisphere and its closest point in a set, Fig. 1b. While this system is currently in the integration phase, experiments were performed using small specimen at different orientations in conventional 3T Siemens Verio scanner. Results The cadaveric knee sample was immobilised in a rotating jig. A 3D_pd_space sequence (TR1300/TE14/FOV250) was used. Images were obtained with 1 mm isotropic resolution. Achieved alignment accuracy was higher than the voxel size. It was proved, Table 1, that using 9 optimal directions is sufficient to achieve required accuracy for the realistic noise levels involved. Table 1 Number of unsuccessfully determined CF orientations as a function of a number of scanning directions and standard deviation of noise Stdev = 2.98

Stdev = 5.4

Stdev = 8.63

N=7

0.12 %

2.1 %

9.3 %

N=8

0.04 %

1.4 %

6.2 %

N=9

0%

0%

0%

Conclusion Successful evaluation of CF orientations is achievable by using only 9 scans at optimally distributed orientations. This is of great significance for in vivo imaging where the overall scan time should be minimised. Appropriate registration/positioning method combined with segmentation has enabled determination of ultrastructure of a reduced number of voxels corresponding to the region of interest. Overall, the method can lead to a significantly better insight into the collagen ultra-structures and improved diagnosis of injury, repair and aging leading to new treatment strategies. References [1] Szeverenyi M, Bydder G (2011) Dipolar anisotropy fiber imaging in a goat knee meniscus. Magnetic Resonance in Medicine 65:463–470. [2] Seidel T, Hammer N, Garnov N, Schneider G, Steinke H (2013) An algorithm for the calculation of three-dimensional collagen fiber orientation in ligaments using angle-sensitive MRI. Magnetic Resonance in Medicine 69:1594–1602.

H. Yousefi1, M. Fatehi2, M. Abbasi1, M. Mohagheghinejad1, R. A. Zoroofi1 1 University of Tehran, Engineering, Tehran, Iran, Islamic Republic of Iran 2 Medical Imaging Informatics Research Center, Tehran, Iran, Islamic Republic of Iran Keywords 3D statistical shape model  Active contour  Point distribution model  Radius and Ulna bones Purpose The Radius and Ulna bones, both with carpus, grow as a unit, and normal growth depends upon a synchronization of the growth of these bones [1]. Statistical models of shape are a promising approach for robust and automatic segmentation of medical image data [2]. This work describes the construction of a statistical shape model of the Radius and Ulna bones. For 3D model-based approaches, however, building the 3D shape model from a training data set of segmented instances of an object is a major challenge and currently remains an open problem. In this study, we propose an active contour image segmentation method, then we create 3D statistical shape model of Radius and Ulna bones for MRI datasets. Methods Here the designed framework takes MR images including Radius and Ulna bones as input and produces the 3D SSM models. The multi-step approach is as following. First we obtain approximate segmented bones that fully separated from other regions. After that we derived a convex hull region around the Radius and Ulna. Finally the estimated convex region is used as an initial mask for active contour algorithm. 1. Radius and Ulna segmentation using active contour: Active contour is minimizing a closed contour to image object boundaries by means of deformation under the influence of image forces, internal forces and external constraint forces. The model is moving under the influence (magnitude and direction) of the internal and external forces in which the tension and the flexibility of the contour manage the contour [3] (see Fig. 1).

Fig. 1 Results of 2D snake segmentation in Radius bone (top) [2], and Ulna bone (bottom). 8, 12, 16, 20 and 24-th coronal slices from left to right 2. Creating the 3-D Statistical Model All shape corresponding methods employ shape registration techniques. Similarity index is used to determine transformation quality in

123

Int J CARS a test. In this method each shape is mapped to the same coordination such that the sum of distances of its corresponding landmarks from average landmarks (average model) should be minimized. Following steps explain how landmarks were extracted: 1. One of the training samples is used as the initial atlas. 2. All training data sets are mapped to the initial atlas. 3. The average shape is derived through averaging registered datasets. 4. The average shape is called an atlas in reference coordination system. To reduce the bias of initial data set, steps 2 to 4 is repeated several times. Produced average shape is used as the initial atlas at each iteration. 5. All registered samples are mapped to Reference Coordinate System (RCS) atlas via a rigid Free Form Deformation FFD mapping. Average of all produced rigid transformations is calculated and is applied to RCS atlas. Obtained average model is called atlas in natural coordination. 6. Marching cubes algorithm is applied on Natural Coordinate System (NCS) atlas which leads to gridding the external surface of Ulna and Radius bones. The numbers of created mesh vertices are used as atlas landmarks. Since each shape consists of many landmarks, and if shapes are represented through their points, vector dimension of each shape is big in comparison to the number of training shapes. So, we should reduce the dimensionality of feature vectors for statistical modeling. A typical method is applying principle component analysis PCA which was used in point distribution model PDM. Finally, this technique introduces a set of modes which represent variations in training data sets. 3. Point Distribution Model (PDM) PDM models the variations of registered shape’s landmarks. The average shape is derived. The statistical analysis is done in this model to estimate landmarks distribution with linear model. This model is called point distribution. In PCA, first eigenvalues and eigenvectors of covariance matrix are calculated. ki represent the variance or model parameters variation which are bi and /i is the matrix including these eigenvectors which are located in its columns. By changing each mode with the amount of +-3Hki the process of allowed variations of statistical model can be observed for a specific parameter. Results Dataset consists of 30 T1-weighted images of hand wrist in coronal view. We applied the active contour algorithm on 27 slices images. Obtained results are in average 92.46 % similar to radiologist’s segmentation in kappa statistic, and their label consistency with radiologist’s segmentation is 96.23 %. Then the contours are aligned in the training data, and the point distribution model known as PDM is produced. As Fig. 2 illustrate, this SSM includes a mean shape of Radius and Ulna, and 5 states of the shape alterations which builds 98 % of the overall complete alterations. We can produce a new shape and rotate, translate, or scale it to match the bones contour in MRI images through PDM. This algorithm works in a way that overall topology of the shape is nearly unchanged. Decimation rate for reducing vertices is equal to 0.98. Since our method is completely based on 3D SSM and the structures are volume; therefore, it is easier to analyses the results, step by step.

123

Fig. 2 Results of mean and first mode variation for Ulna bone (top) and Radius bone (bottom) Conclusion Since 3D data manual approaches for high dimensional data are very time consuming and nearly impossible, creating an automatic method to locate corresponding landmarks is necessary. The major difficulties in building a 3D SSM is establishment of point to point correspondences throughout training set. Manually labeling correspondences on each sample is time consuming and inaccurate even for 2D shapes. Therefore, such a manual approach is impractical for 3D shapes. So, first, using active contour algorithm Radius and Ulna bones is segmented in coronal slices automatically. Then, a statistical model of Radius and Ulna bones is derived. The produced atlas can be used as a primary knowledge of image processing algorithms or be used to detect special irregularities of a group of subjects. References [1] Lim SJ, Udupa JK, Souza A, Torigian D, Jeong YY (2006) A New, General Method of 3-D Model Generation for Active Shape Image Segmentation. SPIE: Medical Imaging, vol. 4298: 48–55. [2] Frangi AF, Rueckert D, Schnabel JA, Niessen WJ (2002) Automatic Construction of Multiple-Object Three_ Dimensional Statistical Shape Models: Application to Cardiac Modeling. IEEE Trans. Med. Image, vol. 21:1151–1166. [3] Yousefi H, Fatehi M, Amian M, Zoroofi RA (2013) A Fully Automated Segmentation of Radius Bone Based on Active Contour in Wrist MRI Data Set. ICBME 20: 42–47

Int J CARS A web-based platform for a high throughput calibration of PET scans D. Zukic´1, Z. Mullen1, D. Byrd2, P. Kinahan2, A. Enquobahrie1 1 Kitware Inc., Medical, Carrboro, N.C., United States 2 University of Washington, Imaging Research Laboratory, Seattle, WA, United States Keywords PET/CT  Calirbration  Phantom  SUV Purpose Positron emission tomography (PET) combined with X-ray computed tomography (CT) has become a standard component of oncology diagnosis and staging over the last decade. Quantitative PET/CT is a valuable tool for assessment of an individual’s response to therapy and for clinical trials of novel cancer therapies because it can measure metabolic changes, which are an earlier indicator of response than anatomical size changes [1]. Thus quantitative PET imaging has an enormous potential to increase the efficiency of trials seeking new therapies. However, the large degree of variability in standardized uptake values (SUVs) arising from inconsistent and non-optimized image acquisition, processing and analysis [2] represents a challenge to the use of PET/CT as a truly quantitative modality. The PET/CT pocket phantom and associated analysis tools described in this paper will permit the measurement and reduction of SUV errors that can improve success rates, reduce time and costs, and improve patient safety in PET imaging trials. The pocket phantom is designed to be simultaneously scanned with a patient to monitor two key sources of SUV bias for small tumors: calibration bias and resolution-loss bias. In addition, having a software infrastructure for high throughput processing of the PET/CT scans is essential in high volume clinical trials. In this paper, we present preliminary work in building a web-based platform for automated PET/CT scan analysis in quantitative imaging. Methods The PET/CT pocket phantom system uses a physical phantom that is scanned with patients and modeling software to estimate scanner calibration bias and reconstructed resolution. Figure 1 shows PET/CT phantoms placed next to an anthropomorphic phantom. Algorithms and associated software tools for analyzing PET/CT images have been shown effective of the expected range of clinical imaging parameters [3].

Fig. 1 Left: PET/CT pocket phantoms placed next to an anthropomorphic phantom. Middle: PET image and a profile of a 1.5 cm Pocket Phantom sphere and a 3 cm sphere (i.e. simulated lesion) placed in the anthropomorphic phantom. Right: fused PET + CT images For automatic deployment of the phantom detection and image characterization, we used a Docker container to easily deploy the 3D Slicer-based modules [4]. A PET image containing a patient and one or more pocket phantoms is fed into the pipeline (Fig. 2). The phantom detector’s first step is thresholding of the image by minimum activity, and condensing leftover parts into connected components. These components (blobs) then get their volume and center of area calculated. Blobs whose radii are outside a specified range are

excluded. Using radius-filtered blobs, all possible 3-blob combinations are constructed. Calculated centers enable pruning the 3-blob combinations by inappropriate sphere distances and usage of noncollinearity as combination’s cost. As lowest cost combinations are declared ‘‘detections’’, other combinations which incorporate the same blobs (conflicting combinations) are eliminated. The optimizer [3] is run on these detections to produce results.

Fig. 2 Initial implementation of Girder-based web framework to run the PET analysis pipeline. This is the review step, in which the user can view the rendering of the image to ensure that right image is submitted to processing Results The software provides these quality assurance capabilities to sites conducting their own clinical trials via the web application, which has three key functionalities: - PET/CT scan upload: Clinical researchers and other users of the platform will upload their datasets using a DICOM transfer and a review process will ensure that the datasets have been correctly identified. Metadata will be automatically extracted and stored in a database, allowing for powerful, flexible search capabilities and displays tailored to the data. - Server-side image analysis: The automated PET/CT scan phantom detection and parameter estimation algorithm are run. - Result reporting: The analysis reporting module will present the key PET image characteristics findings in different formats. To enable scalability, there are two major decoupled services in the deployment: - The web service for user authorization, analysis setup, and displaying of results. This is provided by Girder, a new generation of the open source data management MIDAS platform (Fig. 2). - The processing service that executes the analysis pipeline on the PET/CT data. Since this is computationally-intensive task, it is critical that it can be run in parallel on other machines besides the ones serving the web front-end. The Romanesco execution engine invokes Slicer [4] modules in the Docker container. Conclusion In this paper, we presented a platform built on open source frameworks to load large set of PET/CT scans, conduct server-side image analysis and report results for PET/CT quality assurance. We have adapted our previously reported method to run with very little user intervention by developing automated phantom detection and analysis tools using the Girder framework. To achieve scalability for intense computational tasks, we have built in task distribution and scheduling using the Romanesco execution engine. Docker containers were

123

Int J CARS developed and deployed in order to allow the software to run in a platform independent fashion. This type of pipeline enables high throughput analysis from any site using a pocket phantom for quality assurance, and removes the need for site to acquire any new software or computational resources in order to use the pocket phantom system. References [1] Weber WA, Ziegler SI, Thodtmann R, Hanauske AR, Schwaiger M (1999) Reproducibility of metabolic measurements in malignant tumors using FDG PET. The Journal of Nuclear Medicine 40(11):1771–1777, 1999, PMID: 10565769. [2] Boellaard R (2009) Standards for PET image acquisition and quantitative data analysis. The Journal of Nuclear Medicine 50 Supplement 1:11S-20S, 2009, PMID: 19380405. [3] Byrd D, Wangerin K, Helba B, Liu X, Kinahan P, Avila R (2014) Simultaneous estimation of bias and resolution in PET images with a single long-lived phantom, The Journal of Nuclear Medicine 2014; 55 (Supplement 1):2152. [4] Fedorov A, Beichel R, Kalpathy-Cramer J, Finet J, FillionRobin JC, Pujol S, Bauer C, Jennings D, Fennessy F, Sonka M, Buatti J (2012) 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magnetic resonance imaging. 2012 Nov 30;30(9):1323–41.

Imaging of 188Re filled double-balloon for b-radiation therapy with handheld gamma camera/ultrasound hybrid: a feasibility trial A. van Oepen1, A. Boese1, M. Friebe1 1 Otto-von-Guericke University, Medical Technologies, INKA Chair, Magdeburg, Germany Keywords Rhenium-188  SPECT  Ultrasound  Tracking Purpose Radionuclides can be used after cancer treatment to deliver therapeutic radiation (b-or c-component) to ensure minimal risk of tumor recurrence. They can also be used in combination with external beam radiation therapy (EBRT) as long as suitable applicators are available. Delivering therapeutic radiation by means of minimally invasive methods instead of external beam radiation can lead to better therapeutic outcome and reduce the patient’s trauma and improve the rate of regeneration. The dose is locally delivered and a maximum of healthy tissue can be spared. Since dosimetry and quality assurance are strongly connected to the therapeutic outcome the positioning of the delivery device needs to be placed with diagnostic imaging support and the actual radiation dose measured. Some of the b-radiation emitting radionuclides used, also have a smaller c-component that could be used by appropriate imaging devices for placement and radiation delivery estimation and control. Rhenium-188 is a radionuclide with a b-component (half-life 16.98 h, mean energy 764 keV, max energy 2.12 MeV) for therapeutic irradiation and c-component (155 keV) that could potentially be used for gamma photon imaging [1]. As studies have shown promising results of intravascular radiation therapy (IVRT) treatment with Rhenium-188 agent filled angioplasty balloons [2], we proposed using a double-balloon applicator filled with liquid Rhenium-188 between the inner and outer balloon walls, for delivering therapeutic radiation to minimally invasive accessible post tumor removal areas or solid tumors [3]. Methods Activity measurement and imaging of the Rhenium-188 filled doubleballoon is performed using a handheld gamma camera (CrystalCam, Crystal Photonics GmbH, Berlin, Germany), combined with an infrared tracking system with live image overlay (augmented reality

123

view) of the gamma camera data and a suitable software support (declipseSPECT, SurgicEye GmbH, Munich, Germany). To simulate an area of interest a plastic box (15 9 22 9 15 cm3) filled with water, surrounding the centered double-balloon, was used. The outer balloon had a maximum diameter dout = 49 mm while the inner balloon was inflated with air up to a diameter of din = 40 mm. Hence the spherical liquid filled layer had a diameter of dliquid = 4.5 mm. Verification of the position data was additionally obtained by an ultrasound image of the double-balloon. The Rhenium-188 was delivered through a wolfram shielded applicator. The total activity of the Rhenium was approximately 100 and 200 MBq. Results Utilizing the gamma component of the liquid Rhenium-188, imaging of the double-balloon with the mentioned system and gamma camera could be performed. The position of the radiation source could not be exactly matched with the position obtained by the ultrasound data. The measured source of activity was spherically shaped. Connecting the activity to the imaging data was feasible. Conclusion The measurements of the gamma component could be successfully performed with the handheld gamma camera and tracking system combination. Since the algorithm is based on spherical sources of radiation activity, the resolution of the 4.5 mm diameter Rhenium188 layers was not feasible. Also the voxel size of 5 9 5 9 5 mm3 is limiting the measurement of the position of the rhenium layer. References [1] Wohlfrom M et al. (2001) Endovascular irradiation with the liquid b-emitter Rhenium-188 to reduce restenosis after experimental wall injury. Cardiovasc Res 49(1):169–76. [2] Leissner GG et al. (2011) Endovascular brachytherapy (EVBT) with Rhenium-188 for restenosis prophylaxis after angioplasty of infrainguinal lesions: early experience. Rofo 183(8):735–42. [3] Boese A, Huendorf P, Buck O, Friebe M (2015) Prototype shielded balloon catheter for interventional Rhenium-188 radiation therapy during handheld SPECT/US hybrid imaging/ biopsy procedure. Int J CARS 10(1):183.

Ultra-high definition digital PET/CT for the precise quantitative evaluation of ACL grafts K. Binzel1, R. Magnussen2, C. Kaeding2, D. Flanigan2, W. Wei1, M. U. Knopp3, M. Knopp1 1 The Ohio State University, Wright Center of Innovation, Department of Radiology, Columbus, United States 2 The Ohio State University, Sports Medicine, Columbus, United States 3 Pepperdine University, Sports Medicine, Malibu, United States Keywords PET/MRI  Digital PET  Sports medicine  Ultra-high definition Purpose Injury to the anterior cruciate ligament is common, particularly among young athletes. In cases of a complete tear, surgical placement of a graft is often required in order to restore stability and function to the knee. Graft loading prior to adequate healing greatly increases the risk of graft failure. Graft healing through the process of ligamentization is not well delineated by anatomic MRI alone. PET/CT molecular imaging in combination with standard or care MRI has proven feasible for evaluation of ACL graft healing following surgery. The recent development of digital PET/CT systems now enables improved system resolution, Time of Flight timing resolution, as well as increased reconstruction matrix sizes with smaller voxel volumes. We assessed the gain in quantitative accuracy from using ultra high definition digital PET for the evaluation of graft metabolic activity.

Int J CARS Methods 10 patients post-surgical ACL graft placement underwent standard of care MRI on a 3T Ingenia CX. PET/CT imaging was performed on a pre-commercial release next generation digital PET/CT system, Vereos (dPET). An in-house fabricated mold of the MR knee coil was used during PET to ensure identical positioning between image sets. A single bed position centered on the knees was acquired following a 3 mCi [111 MBq] 18F-FDG injection. Listmode data collected 60–75 min post-injection were reconstructed using both the standard 144 9 144 matrix settings resulting in 4 mm3 voxel volumes as well as a 576 9 576 matrix with 1 mm3 voxel volumes, ultra-high definition protocol. Patients were grouped by time since surgery, from less than 6, 6–12, 12–24, and more than 24 months since surgery. SUVmax was measured in the proximal, middle, and distal portions of the graft, femoral and tibial tunnels, the posterior cruciate ligament (PCL), and quadriceps muscle for reference. Matched ROIs were drawn in the contralateral knee. The percent difference from the 4 mm3 image measurements was calculated for each matched 1 mm3 image measurement. Results SUVmax was found to increase with the use of an ultra-high definition reconstruction protocol. The increase was most substantial in areas of increased metabolic uptake, the graft and bone tunnels, in patients with more recent surgery. The background areas of PCL and muscle, as well as grafts which had been in place longer, had overall lower metabolic activity and therefore did not demonstrate significant (p [ 0.1) changes in SUVmax with the 1 mm3 image sets. This is in agreement with phantom experiments validating the improved recovery coefficients in small, activity intense areas with the use of larger reconstruction matrix sizes. Conclusion PET/MRI imaging of the knee, especially ACL grafts, benefits from higher resolution reconstruction due to the improved quantitative precision enabled by a reduction of partial volume effects. The dPET platform provides the improved Time of Flight timing resolution and count statistics required for ultra-high definition imaging of small structures, such as those in the knee. Where conventional PET has been limited in the ability to accurately visualize small, metabolically active areas, dPET is capable of overcoming these hurdles, even with low dose imaging approaches. Ultra-high definition dPET also enables better voxel-wise registration to MRI, further improving quantitative accuracy. Providing digital computer-based quantitative analysis in conjunction with clinical evaluations of graft healing can help guide physicians in making rehabilitation and return to sport decisions, lowering the chance of graft failure (Fig. 1).

Fig. 1 Sample images from a patient between 6 and 12 months since surgery. PET, MR, and fused images show excellent co-registration between image sets. The view through the center portion of the graft shows the excellent level of visual quality achieved with the ultra high definition reconstruction protocol. Precise quantification is also attained through use of optimized reconstruction settings with the digital PET acquisition. Here, increased metabolic uptake can be observed in the graft, while the bone tunnels appear to already have uptake approaching that of healthy tissues

A system for multispectral real-time fluorescence guided cystoscopy N. Dimitriadis1,2, M. Theuring1, B. Grychtol1, M. Kriegmair3, N. Deliolanis1 1 Fraunhofer, Project Group for Automation in Medicine and Biotechnology, Mannheim, Germany 2 University of Heidelberg, Department of Physics and Astronomy, Heidelberg, Germany 3 University Medical Centre Mannheim, Department of Urology, Mannheim, Germany Keywords Fluorescence  Multispectral  Imaging  Cystoscopy Purpose In the past years a variety of imaging methods to plan and guide surgical procedures have appeared. Though, the majority of surgical interventions is still based on the surgeon’s visual perception. Seminal research in the field of molecular imaging has resulted in fluorescence imaging methods that highlight tissue anatomy, function and pathology. Novel targeted fluorescent contrast agents in clinical translation are administered in the body to circulate, target and bind to tumors, nerves, inflammation, angiogenesis, metabolism, etc. Fluorecence imaging aids both the diagnostic and the interventional procedures by allowing the medical doctors to easily differentiate tissues beyond the capacity of the standard color vision and conventional imaging systems. Yet current systems are capable to display either fluorescence by illumination with light being spectrally confined to excite fluorescence or to display a conventional color image by illuminating the tissue with an alternative light. But there are limitations in the functionality of the systems which do not allow to visualize conventional color images with fluorescence information. Herein, we present a novel imaging and processing technology that enables conventional color imaging along with multiple fluorescent components for image guided intervention. Methods For application in bladder cystoscopies we have adapted our imaging system to a rigid transurethral endoscope. The imaging optics in front of the HD camera has been modified by placing a multiple bandpass filter that blocks the light to be detected in several spectral bands. The illumination system consists of multiple LEDs that are filtered by bandpass filters and operates sequentially in two phases. In phase 1, the tissue is illuminated with light matching the spectral bands of the filter placed in front of the sensor. In phase 2, the tissue is illuminated with light that is completely blocked by the filter in front of the camera. Thus fluorescence can be detected in this second phase, because the excitation light is filtered. If the two phases are repeated fast enough, the system allows a fluent real-time acquisition of both, fluorescence and color images. Results We have demonstrated the capability of the cystoscopic imaging system to record and multispectral fluorescence images od multiple fluorophores. The multispectral fluorescent images can be unmixed after a calibration procedure with a linear unmixing algorithm spearating the individual dyes. Linear unmixing allows a very efficient implementation in the processing unit, but it is sensitive to noise. Conclusion We demonstrate that our real-time multispectral imaging technology improves the visualization of fluorescence guided interventions. Even though the presented system aims at bladder cystoscopy, the modular design allows the adaptation to other systems as well, for example, neurosurgical microscopes. Surgeons will benefit from the fluent realtime visualization of conventional color images enhanced with information from fluorophores. The possibility to record fluorescent emission light of multiple fluorophores passing through a multibandpass filter with a multi-channel camera enables the distinction between different fluorescent contrast agents. Additionally, multispectral imaging has the capacity to separate the specific fluorescence signal coming from the contrast agents against unspecific tissue

123

Int J CARS autofluorescence, thus enhancing the sensitivity and specificity in fluorescence guided interventions.

Optical coherence tomography (OCT): The missing link in gastrointestinal imaging? M. Rehberger1, D. Wilhelm2, K.- P. Janssen2, M. Gu¨nther3, M. Witte1, N. Ko¨nig1, R. Schmitt1,4 1 Fraunhofer Institute for Production Technology IPT, Aachen, Germany 2 Klinikum rechts der Isar der TUM, Surgery, Munich, Germany 3 Fraunhofer MEVIS, Bremen, Germany 4 RWTH Aachen University, Aachen, Germany Keywords Optical coherence tomography  Imaging  Colon adenocarcinoma  Oncology Purpose Clinical treatment decisions in surgical oncology rely on accurate information about the histological and cellular architecture of a given tumor lesion, routinely gathered by histopathological assessment of biopsies or resected tissue. This is a relatively time-consuming and costintensive procedure consisting of fixation, staining and visual examination by conventional light microscopy. Vital, staining-free, highresolution and rapid imaging modalities could offer a dramatic advantage in terms of time, cost-effectiveness and reduced hands-on time. We propose here optical coherence tomography (OCT) as a noninvasive optical imaging modality ideally suited to fit these needs. OCT imaging relies on light scattering from the sample and subsequent optical interference. It allows assessing the histological morphology with high axial resolution without requiring contrast agents. Although already implemented in other fields, its application in gastrointestinal imaging is pending and its full potential has not yet completely been explored. As a proof-of-concept study, and to investigate its possible imaging qualities, a spectral domain OCT (SD-OCT) set-up is used to evaluate its feasibility for clinical purposes. Methods As shown in Fig. 1, we use a SD-OCT based on a fiber-coupled Michelson interferometer configuration. Due to its spectral bandwidth (FWHM) of 100 nm at 1325 nm central wavelength, the system allows for a high axial resolution of \ 7 lm and a measurement range of 2.5 mm. In addition to that the lateral resolution is \ 16 lm at 10 mm lateral measurement range respectively. The OCT set-up is able to characterize semi-transparent specimen like tissue in a non-destructive way and is free of ionizing radiation and contrast agents. A scanning unit allows for acquiring volumetric datasets that are exported in an open source raw data format for enhanced post-processing.

Image data visualization and analysis was performed in MeVisLab (ML), a powerful, modular framework for image processing research and development with a special focus on medical imaging [1]. It allows fast integration and testing of new algorithms and the development of clinical application prototypes. OCT image data was converted to the ML data format and displayed in three orthogonal views as well as by volume rendering. Tissue from n = 3 patients diagnosed with colorectal cancer was surgically resected at the Dept. of Surgery at TUM. Moreover, inflamed colon mucosa from a patient with inflammatory bowel disease was analyzed. Tumor tissue and adjacent normal mucosa were either shock-frozen immediately, or fixed in formalin over night and placed in 70 % ethanol. Moreover, tissue samples from transgenic mouse models for colon cancer from n = 2 mice (ZPF animal facility at TUM, strain Apc1638N) were used along the same protocol. Imaging with OCT was carried out after thawing the shock-frozen samples, or directly imaging ethanol-fixed samples. Results The measurements were performed at Fraunhofer IPT using the OCT set-up shown in Fig. 1. Human tissue samples were analyzed by the scanning unit in three dimensions, 8 mm by 8 mm laterally and 2.5 mm axially. The penetration depth in tissue strongly depends on absorption and scattering properties in the infrared for each specimen. Thus, the OCT signal decays exponentially with increasing imaging depth. As depicted in Fig. 2 (right), the samples were probed ca. 1 mm in depth, providing adequate structural information to compare to histological stained sections. For lossless data transmission the VTK data format was chosen and the tissue datasets imported into ML. An adjustment of brightness and greyscale mapping offers improvements regarding the image contrast of structural information slightly above the noise level (compare (A) and (B) in Fig. 2).

Fig. 2 OCT imaging on normal colon mucosa (A) and colon adenocarcinoma (B) with addition of HE stained sections. The OCT workup included a 3D reconstruction (upper-left border) as well as multi-planar visualization including orthogonal oriented planes

Fig. 1 Broadband spectrum is emitted by a superluminescence diode. The light is split by a fiber coupler into a reference and a probe beam. In a scanning unit the probe beam is deflected by set of galvo scanners to generate the OCT scans. The interference of the probe and reference beam is detected with a spectrometer and analyzed on a computer

123

We demonstrate the feasibility of OCT imaging on tissue samples from both preclinical animal models and human patient samples of colon cancer and normal colon mucosa, both from fresh-frozen as well as formalin-fixed specimens. Tissue architecture was well distinguishable in OCT imaging, providing sufficient information to differentiate between the respective intestinal layers and to identify tissue abnormalities and pathologic features. The quality of tissue preservation and procession mainly affected imaging quality.

Int J CARS Conclusion OCT imaging allows for rapid and staining-free assessment of colon tissue morphology providing a resolution almost sufficient to identify the underlying pathology. Future benefits comprehend the miniaturization of the probe and the further optimization and adoption of OCT for use in gastrointestinal medicine. The OCT measurements provided by Fraunhofer and the visualization by means of the ML imaging software even now offers a quality that is far beyond available nondestructive imaging systems like ultrasound or MRI. The OCT approach provides a technical platform which can be adapted with its physical setting and parameters to individual application requirements. This technical flexibility opens a wide variety of new application perspectives. The limitation in tissue penetration depth of optical imaging systems is present for OCT as well, but it shows the achieved imaging section is attractive for in vivo applications. Absence of contrast agents and ionizing radiation reduce necessary efforts in cost and time and give the ability for high repeatability. It has to turn out whether OCT can compete with the diagnostic accuracy of conventional microscopic analysis of resected tissue. With its optical resolution beyond 10 lm and its in vivo applicability, OCT might be the missing link between conventional endoscopy and microscopy. Further application of OCT imaging especially in fields of interventional medicine seems attractive. Reference [1] Ritter F, Boskamp T, Homeyer A, Laue H, Schwier M, Link F, Peitgen H–O (2011) Medical Image Analysis: A visual approach. IEEE Pulse. 2(6):60–70.

Siemens AG, Berlin, Germany) and a medical monitor (RadiForce GS220;, EIZO Co., Ltd., Hakusan, Japan), whereas the images of RMI156 phantom was taken using a mammography system (MAMMOMAT Novation DR,Siemens AG) and a medical monitor (RadiForce GX540;, EIZO Co., Ltd., Hakusan, Japan). The Burger phantom image was acquired using a tube voltage of 75 kV, sourceimage distance of 105 cm, and doses of 10, 1.0, and 0.5 mAs. The RMI156 phantom image was acquired using a tube voltage of 27 kV, Mo/Mo (target/filter), and a dose of 80 mAs. Six certified radiological technologists evaluated the Burger phantom image and ten certified radiological technologists evaluated the RMI156 phantom image with and without ScSR processing (Fig. 1). The visible minimum diameter and depth of mimic lesions were determined, and contrast-detail (CD) curves were created at each exposure dose. Thus, a subjective validation was performed based on image quality figures (IQFs), which were calculated based on the resulting C-D curves. An IQF is the integration of the minimum diameter of the signal at each contrast and is calculated as follows: IQF = R [i = 1, n] Ci 9 Di min, where Ci, Di, and n are the depth and diameter of each mimic lesion and the number of steps, respectively. A smaller IQF indicates a better image quality [2].

Evaluation of X-ray images using sparse coding super-resolution processing Y. Miyasaka1, S. Sanada2, M. Higashi3, M. Ogaki3, M. Kita3, Y. Matsuda4, K. Imamura4 1 Kanazawa University, Division of Health Sciences, Graduate School of Medical Sciences, Kanazawa, Japan 2 Kanazawa University, Department of Radiological Technology, School of Health Sciences, College of Medical, Pharmaceutical and Health Sciences, Kanazawa, Japan 3 EIZO corporation, R&D, Visual Technologies (ASIC), Hakusan, Japan 4 Kanazawa University, Faculty of Electrical and Computer Engineering Institute of Science and Engineering, Kanazawa, Japan Keywords Image quality  Digital radiography  Contrast-detail  Radiation Measurements.Inc.156 Purpose Sparse coding super-resolution (ScSR) processing has been reported to improve the resolution characteristics for general images. In this study,we evaluated the impact of Burger and RMI156 phantoms on image resolution using ScSR processing. Methods ScSR processing divides target images into high and low- frequency components. It has a so-called ‘‘dictionary’’, which the sharp and deteriorated image components of various image features. Similar features of the input image were retrieved from the deteriorated image components in the ‘‘dictionary’’, and were then replaced with the sharp image components. Finally, the sharpness of the original input image was improved [1]. To evaluate image quality, the image of Burger phantom was taken using an indirect flat panel detector equipped with an X-ray fluoroscopy device (AXIOM Luminos dRF;,

Fig. 1 A burger phantom image is normal image (a), zoomed of the a (b) and processed image of the a (c). A RMI156 phantom image is normal image (d), zoomed of the d (e) and processed image of the d (f) The RMI 156 phantom was evaluated by the pair comparison method. The matrix size of the RMI156 phantom was varied by normal (2560 9 3328), half (1280 9 1664) and quarter (640 9 832). The study team performed ScSR processing to three images and ultimately acquired six images. Two images were then selected from the six candidate images and subsequently compared. This operation was performed 30 times. The items evaluated were good sharpness level, low noise level and preference. A larger count indicated good sharpness level, low noise level and preference [3]. Results The resolution of the visible area was improved by the inclusion of the Burger phantom in the degraded image during ScSR processing. At doses of the 10, 1.0 and 0.5 mAs, IQFs of the processed and original images were 81.3 and 86.4, 91.04 and 100.4, 95.7 and 120.1, respectively (Fig. 2). The effects of ScSR processing were confirmed at all doses by smaller IQFs of processed images.

123

Int J CARS Discovering clusters in pathologic cardiac morphology: MR-based hierarchical 3D shape clustering of surgically repaired aortic arches J. L. Bruse1, K. McLeod2,3, E. Cervi1, G. Biglino1, T.- Y. Hsia1,4, M. Sermesant3, X. Pennec3, A. M. Taylor1,4, S. Schievano1,4 1 UCL Institute of Cardiovascular Science & Great Ormond Street Hospital for Children, Centre for Cardiovascular Imaging, London, Great Britain 2 Simula Research Laboratory, Cardiac Modelling Department, Oslo, Norway 3 Inria Sophia Antipolis-Me´diterane´e, ASCLEPIOS Project, Sophia Antipolis, France 4 MOCHA Collaborative Group, Modeling of Congenital Hearts Alliance (MOCHA) Group, London, Great Britain Fig. 2 Comparison of IQF calculated from contrast-detail curves for normal images and processed images (n = 6) ScSR processing improved the RMI156 phantom of the resolution characteristics of fibrous tissue by more than 0.40 mm, of calcified tissue by more than 0.24 mm, and mass by more than 0.20 mm. Counts of ‘‘good sharpness level’’ for the original and processed images of the RMI156 phantom were 85 and 93 for the normal matrix size, 28 and 57 for the half-matrix size, and 2 and 35 for the quarter-matrix size, respectively. The counts of ‘‘low noise level’’ for the original and processed images were 7 and 23 for the normal matrix size, 83 and 43 for the halfmatrix size, and 92 and 52 for the quarter-matrix size, respectively. The counts of ‘‘preference’’ for the original and processed images were 77 and 84 for the normal matrix size, 36 and 64 for the half-matrix size, and 12 and 27 for the quarter-matrix size, respectively (Table 1). For the most part, normal matrix size did not realize significant differences between the original and processed images; however, half-matrix and quarter-matrix sizes tend to exhibit significant differences. Conclusion ScSR processing using the appropriate process parameters and ‘‘dicTable 1 Counts of ‘‘good sharpness’’, ‘‘Low noise’’ and ‘‘preference’’ of each matrix size Matrix Size

Good Sharpness

Low Noise

Original

processed

Original

Normal

85

93

Half

29

57

2

35

92

Quarter

Preference processed

Original

processed

7

23

77

84

82

43

36

64

52

12

27

tionary’’ is promising for improving the resolution characteristics of X-ray images. References [1] Yang J, Wright J, Huang T, Ma Y (2010) Image SuperResolution via Sparse Representation. IEEE Transactions on Image Processing 19 (11) : 2861–2873. [2] Tanaka R, Ichikawa K, Matsubara K, Kawashima H (2012) Review of a simple noise simulation technique in digital radiography. Med Phys 5 (2) : 178–185. [3] Nishihara S, Harada A, Takahashi M, Koike M, Watanabe S, Kohama C, Sanada T (2012) Significance of the Visual Evaluation Method Using the Two-visit Method (Two-sample Preference Test Suggested by Ferris). Med Phys 59 (1): 130–135.

123

Keywords Hierarchical clustering  Statistical shape analysis  Cardiovascular MR  Aorta Purpose Cardiovascular magnetic resonance (CMR) imaging provides detailed 3D anatomical shape information, and is increasingly used for diagnosis and pre- and post-surgical assessment of complex cardiac morphology. This leads to growing medical image databases, allowing the use of computational tools from the field of data mining to analyse and structure the bulk of shape information. Here, we combine a statistical shape modelling framework (SSM) with hierarchical clustering techniques in order to detect previously unknown shape sub-groups (i.e. clusters) of surgically reconstructed aortic arches in patients with hypoplastic left heart syndrome (HLHS). Children with HLHS are typically born with an underdeveloped and small aorta, which needs to be surgically enlarged and reconstructed in order to allow sufficient blood supply to the lungs and body [1]. While resulting aortic arch shapes vary significantly in shape, associations between arch morphology and functional or clinical outcome have not been explored in detail. We hypothesised that our method can cluster together similarly shaped aortas, aiming to unveil functional similarities within, and differences between shape clusters. Methods This is a retrospective study based on routine CMR data of 40 HLHS patients acquired 3.08 ± 1.12 years after surgical arch repair. Parents gave informed consent for research use of the data. Aortas were segmented manually (Mimics, Materialise, Leuven, Belgium) from CMR whole-heart data (1.5T Avanto scanner, Siemens Medical Solutions, Erlangen, Germany) and 3D surface meshes of the arches were created. Meshes were rigidly aligned using an iterative closest point algorithm [2]. The aligned meshes constituted the input for the SSM framework Deformetrica [3], which essentially computes the mean 3D shape (i.e. template) and the deformation vectors, registering the template towards each subject shape— without the need for landmarking (Fig. 1). Thus, each patient’s arch morphology is characterised by its unique set of deformation vectors. The sets of deformation vectors were combined into a deformation matrix, which was used as input for a hierarchical clustering algorithm (MATLAB, Natick, MA) that does not require specifying an expected number of clusters prior to calculation. Results were visualised as a dendrogram, where objects are grouped together based on their arch shape similarity. By cutting the dendrogram horizontally, the data were partitioned into naturally occurring shape clusters [4].

Int J CARS

Fig. 1 Aortic arch surface models of 40 HLHS patients were reconstructed from CMR data (a, random order). The non-parametric SSM computed the anatomical mean shape of the population (template, b) and its surface deformation vectors Ui towards each subject shape i. The combined matrix of all subject-specific deformation vectors contained all shape information and constituted the input for the hierarchical clustering algorithm Found clusters were compared with regard to traditional morphometric parameters (surface to volume ratio SVol, centreline length CLL and tortuosity CLT and average indexed arch diameter along the centreline iDav) and CMR-derived parameters characterising cardiac function (ventricular ejection fraction EF, indexed end-diastolic volume iEDV and indexed cardiac output iCO) using Kruskal–Wallis tests (IBM SPSS Statistics, Chicago, IL). Results The clustering yielded two main aortic arch shape clusters (A, B, Fig. 2), each splitting into two larger subgroups (1, 2 and 3, 4, Fig. 2). Two outlying shapes formed each a single cluster as part of cluster B (Fig. 2). The most outlying subject, least similar to any other shape, presented a highly dilated and long aortic root with a very slim arch continuation. Distributions of the measured morphometric parameters SVol (p \ .001), CLT (p = .001) and iDav (p = .009) were significantly different between the four main shape clusters.

In terms of functional parameters, distributions of iEDV were significantly different (p = .029) between the four groups, with lower values occurring in cluster 1 and higher values occurring in cluster 4. The latter contained subjects with relatively wider arches and with a markedly dilated aortic root compared to the ascending aorta and arch continuation. Distributions of EF (p = .283) and iCO (p = .138) did not differ significantly. Conclusion We present the first application of 3D statistical shape modelling combined with hierarchical clustering techniques to a population of surgically repaired aortas in patients with HLHS. By visual assessment, similar arch shapes were clustered together correctly, which was supported by traditional morphometric parameters being different between the four main sub-groups. Interestingly, three out of four subjects who deceased were clustered together in one shape cluster. Further, the most outlying patient in terms of arch shape required a heart transplant. Those findings, together with the highlighted differences in iEDV between clusters suggest that functional and outcome data may be associated with arch shape features detected via the proposed deformation-based shape clustering. Studies in a larger cohort, including more clinically relevant data, are necessary to support our findings. Yet, particularly with the perspective of automating the segmentation process, our clustering framework may provide an attractive research tool as well as a platform for the retrieval of subjects with similar morphology from medical image databases for clinical decision support. Combining advanced 3D statistical shape modelling with data mining techniques could help facilitate and improve risk assessment and treatment planning in complex lesions. References [1] Feinstein JA, Benson DW, Dubin AM, Cohen MS, Maxey DM, Mahle WT, Pahl E, Villafan˜e J, Bhatt AB, Peng LF, Johnson BA, Marsden AL, Daniels CJ, Rudd NA, Caldarone CA, Mussatto KA, Morales DL, Ivy DD, Gaynor JW, Tweddell JS, Deal BJ, Furck AK, Rosenthal GL, Ohye RG, Ghanayem NS, Cheatham JP, Tworetzky W, Martin GR: Hypoplastic Left Heart Syndrome: Current Considerations and Expectations. Journal of the American College of Cardiology. 59, S1-S42 (2012). [2] Besl PJ, McKay ND: A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence. 14, 239–256 (1992). [3] Durrleman S, Prastawa M, Charon N, Korenberg JR, Joshi S, Gerig G, Trouve´ A: Morphometry of anatomical shape complexes with dense deformations and sparse parameters. NeuroImage. 101, 35–49 (2014). [4] Hastie T, Tibshirani R, Friedman J: The Elements of Statistical Learning. Springer New York, New York, NY (2009).

Radiological evidence of 3D internal structure in hip osteophytes G. Venne1, S. Ryan2, J. F. Rudan3, R. E. Ellis4,2,3,1 1 Queen’s University, Biomedical and Molecular Sciences, Kingston, Canada 2 Queen’s University, School of Computing, Kingston, Canada 3 Queen’s University, Department of Surgery, Kingston, Canada 4 Queen’s University, Department of Mechanical and Materials Engineering, Kingston, Canada Keywords Osteoarthritis  Osteophyte  Computed tomography  Bone density Fig. 2 Dendrogram revealing the natural grouping within the shape data. Apart from two outlying subjects, four main shape sub-groups were identified

Purpose The presence and degree of maturity of osteophytes, along with joint space narrowing, are the main radiographic criteria for currently

123

Int J CARS accepted diagnosis and grading of hip osteoarthritis; osteophytes are both currently accepted [1] and widely accepted [2] signs. Osteophytes are specifically correlated with pain reported by patients [1]. Osteophytes can occur in non-symptomatic joints, in joints with no other observable alterations, and in early stage OA [3][4]; it is unclear whether osteophytes arise because of pathological molecular alterations or from natural tissue remodeling processes due to mechanical modifications [3]. Osteophytes are known to constitute confounding factors in image processing, particularly for planning of computerassisted procedures [5]. This study evaluated the 3D internal structure of mature hip osteophytes using microscopic computed tomography (lCT). We hypothesized that mature femoral head osteophytes have organized internal structure. Knowing this internal structure may help in the design and evaluation of future image processing algorithms. To quantitatively evaluate the organization and internal structure from femoral head osteophytes, a detailed 3D radiological morphometric analysis was performed. Osteophytes were compared to two regions of the femoral head: subcortical bone of the overall femoral head, and the substructure known as the primary compressive group. The compressive group, found mainly in the femoral neck, is known to have a highly organized internal structure. Methods With approval of the relevant IRB, the femoral head and neck were retrieved from 16 patients who underwent total hip arthroplasty. lCT scans (speCZT, GE, US) were acquired with isotropic voxels, reconstructed at 100 lm and sliced in the coronal plane. Each volume of interest (VOI) excluded cortical bone. 3D morphometric analysis was performed using commercial software (Skyscan, BE). For each osteophyte, the VOI included relevant internal structures. For the overall femoral head, the VOI included all trabecular bone. For the primary compressive group, VOI’s were rectangular prisms in the neck and shaft. Visualizations of representative volumes are in Fig. 1.

Fig. 1 3D reconstruction of internal structure of a) Femoral head b) Osteophyte c) Primary compressive group Images were thresholded based on histograms. Numerical measures to evaluate each VOI were selected based on their wide acceptance, excluding measures that were deemed contentious or that might be unreliable on the small VOI of osteophytes. The three measures were: (1) bone volume fraction, the relative number of above-threshold voxels in each VOI; (2) degree of anisotropy, where 1 indicated isotropy and higher values indicated higher anisotropy; (3) connectivity density as computed from the Euler characteristic of each voxel, with larger numbers indicating higher connectivity. The set of osteophyte VOI’s was compared to the femoral-head VOI’s and compressive-group VOI’s separately. Comparisons were made with unpaired 2-sided t tests. Data are presented as mean ± standard deviation. Results Findings, and statistical significance of comparisons, are summarized in Tab 1. Details are: (1) Bone volume fraction. Overall head fraction was 34.8 % ± 3.8 %; osteophyte fraction was 29.5 % ± 7.5 %;

123

compressive-group fraction was 39.4 % ± 5.8 %. Osteophyte fractions were significantly different from the overall head (p) and from the compressive group (p). This measure indicated some internal structure. (2) Degree of anisotropy. Overall head anisotropy was 1.6 ± 0.2; osteophyte anisotropy was 1.9 ± 0.4; compressive-group anisotropy was 2.8 ± 0.4. Osteophyte anisotropy was significantly different from the overall head (p) and from the compressive group (p). This measure indicated substantial internal organization that was confirmed by visual inspection of the lCT images. (3) Connectivity index. Overall head connectivity was 2.0 ± 0.6; osteophyte connectivity was 2.7 ± 0.9; compressive-group connectivity was 1.9 ± 0.8. Osteophyte fractions were significantly different from the overall head (p) and from the compressive group (p). This confirmed that the osteophytes had a substantial internal structure with considerable trabecular-like branching. Overall, a mature hip osteophyte had a definite internal structure that was intermediate between the overall femoral head and the primary compressive group. Conclusion The bone volume fraction indicated that osteophytes were less dense than the overall femoral head but were neither empty nor were they completely calcified. The degree of anisotropy indicated that osteophytes had more anisotropic structure than the overall femoral head and were less anisotropic than the highly structured compressive group. The connectivity index indicated that osteophytes had more connected trabecular structure than did the overall femoral had and the primary compressive group. This is a first report on 3D morphometric analysis of hip osteophytes. We found that, far from being amorphous growths around a joint, femoral-head osteophytes had internal structure with an impressive degree of alignment. The osteophyte internal structure seems to be intermediate between that of the overall femoral head and that of the highly loaded primary compressive group. Traditional literature on osteophytes does not mention any internal structure whatsoever, whereas we found a structure similar to trabecular bone. We speculate that, because osteophytes arise in regions with compressive or tensile loading, such loading may be a contributor to their genesis and maturation. These findings may be useful in the classification of one of the most important ways of grading osteoarthritis, and in developing or assessing image processing algorithms for computer-assisted surgery of joints. Further study is needed to determine when and how the internal structure of an osteophyte arises (Table 1). Table 1 Findings, and statistical significance of comparisons

Bone volume fraction

Femoral Head

Osteophytes

Compressive Group

34.8 ± 3.8

29.5 ± 7.5

39.4 ± 5.8

p \ 0.002

Degree of anisotropy

1.6 ± 0.2

Connectivity density

2.0 ± 0.6

p \ 0.001 1.9 ± 0.4

p \ 0.001 p \ 0.001

2.8 ± 0.4 p \ 0.001

2.7 ± 0.9

1.9 ± 0.8 p \ 0.003

References [1] Arden NK, Lane NE, Parimi N, Javaid KM, Lui LY, Hochberg MC, Nevitt M (2009) Defining incident radiographic hip osteoarhtritis for epidemiologic studies in women. Arthritis Rheum 60(4): 1052–1059. doi:10.1002/art.24382.

Int J CARS [2]

[3]

[4]

[5]

Kellgren JH, Lawrence JS (1957) Radilogical assessment of osteo-arthritis. Ann Rheum Dis 16:494–502. doi:10.1136/ard. 16.4.494. van der Kraan PM, van den Berg WB (2007) Osteophytes: relevance and biology. Osteoarthr Cartil 15:237–244. doi: 10.1016/j.joca.2006.11.006. Roemer FW, Jarraya M, Niu J, Silva J-R, Frobell R, Guermazi A (2015) Increased risk for radiographic osteoarthritis features in young active athletes: a cross-sectional matched case–control study. Osteoarthr Cartil 23:239–243. doi:10.1016/j.joca. 2014.11.011. Kunz M, Balaketheeswaran S, Ellis RE, Rudan JF (2015) The influence of osteophyte depiction in CT for patient-specific guided hip resurfacing procedures. Int J Comput Assist Radiol Surg 10(6):717–726. doi:10.1007/s11548-015-1200-7.

Fig. 1 Illustration of a patient scan slice: baseline (left) and a synthetic repeat scan with a ‘‘tumor’’ added (right)

Repeat CT scanning dose reduction without image quality loss: preliminary results N. Shamul1, L. Joskowicz1 1 Hebrew University of Jerusalem, Jerusalem, Israel Keywords CT  Image processing  Low dose scanning  Sparse scanning Purpose Computer Tomography (CT) scans are very common and are used in a wide variety of clinical scenarios. The main disadvantage of CT imaging is that it exposes the patient to ionizing radiation (X-rays), which has been shown to be harmful. Repeat CT scanning presents significant, untapped opportunities for dose reduction and optimization. In repeat CT scanning, a patient is scanned some time after a baseline scan was acquired. It is very common in many clinical situations and includes multi-phase scanning, intra- and post-procedural scanning, and follow-up scanning, especially in oncology. An important observation is that in many cases, the imaging changes in the repeat scan are confined to a few, small regions, while the remaining regions are very similar to those of the baseline scan. We have developed a new algorithm for reducing the repeat scan dose by scanning only the changed regions and using the baseline scan to complete missing information and produce the repeat scan without compromising its quality. Methods We have developed a repeat scan method that optimizes the X-ray dose needed during scanning as a function of the detected changes in the patient since the baseline was acquired. It first aligns the baseline scan to the patient (Fig. 1) by registering the full baseline sinograms to partial sinograms scanned selectively from a small set of angles [1]. Next, for each angle scanned, it finds displacements at which the rays passed through changed areas. These areas in the sinogram signal are identified based on peaks in the difference between the baseline and repeat sinograms at the given angle. The binary masks produced for each angle are back-projected to generate a likelihood map corresponding to image locations. Each element in the map holds the likelihood that the location was changed (Fig. 2, left). Using this map, a scan mask is generated for the remaining angles. Only displacements deemed likely to pass through a changed area are chosen (Fig. 2, right). The remaining scan angles are then acquired according to the mask, producing a partial repeat scan sinogram with information for only select displacements from each view. The missing data is completed using the full baseline sinogram. Standard image reconstruction is used to obtain the final image.

Fig. 2 Illustration of the output for the slice in Fig. 1. Left: likelihood map superimposed on the slice. The yellow area indicates the region found to have the highest likelihood of being changed. Right: the final scan mask for that slice, showing which elements in the sinogram should be acquired in the repeat scan. Light blue indicates rays that actually pass through the changed region, and yellow indicates rays chosen by our method but not by the optimal scan mask We tested our method on simulated pairs of baseline and repeat scans (Fig. 1). Changes were generated by adding synthetic ‘‘tumors’’—cuboids or ellipsoids with non-uniform gray levels (std = 0.1). The location, size and mean gray level of each tumor were chosen randomly for each pair. Pairs varied in the type of change introduced in the repeat scan and in their difficulty level. Changes simulated clinical scenarios, including the occurrence of a new tumor, the expansion or shrinkage of an existing tumor and a change in a tumor’s gray level. Four levels of difficulty were defined by the tumors’ size and contrast. Results We tested our method with two studies. In the first, the datasets were based on the 3D Shepp-Logan head phantom (256x256x256 voxels). A total of 480 trials were run, 30 for each combination of difficulty and type of change. In the second study, a real patient head CT scan served as the baseline. In this study, the difference between the two scans was always the appearance of a new tumor. A total of 120 trials were run, 30 for each difficulty. In all experiments, the scan mask was generated using only 12 evenly distributed scan angles. We measured both the accuracy and radiation dose reduction of our method. Accuracy was defined as the identification of the changed regions; we calculated the precision and recall with respect to the data in the full repeat sinogram that corresponded to changed areas. We defined the dose reduction as the percent of rays needed for a full scan that were selected by our method for the partial repeat scan. The optimal dose reduction was found based on the actual locations of the changes, allowing us to measure how near-optimal our output was.

123

Int J CARS The mean recall was over 98 % for all but the most difficult input, in which the tumors were small and with low contrast. In 6 out of 150 trials with this difficulty level no change was found, yet the mean recall remained above 93 % for all types of changes. The mean precision in changed slices depended on the tumors’ size—for medium sized tumors (8–33 voxels in each dimension) it ranged from 33 % to 46 %, and for small tumors (4–16 voxels in each dimension) it ranged from 20 % to 30 %. The dose reduction was consistent for all datasets, and the difference between our results and the optimal dose ratio was always \ 8.5 % of the dose required for a full scan. Conclusion We have developed a new method for optimizing the dose of a repeat CT scan using information from the baseline scan. Our method is fully automatic and is complementary to existing dose reduction methods. We have shown that even in highly challenging cases our method successfully identifies the locations in which the object has changed, and generates high quality reconstructions with only 8.5 % of a full radiation dose more than the optimum without compromising the image quality. References [1] Medan G, Kronman A, Joskowicz, L. Reduced-dose patient to baseline CT rigid registration in 3D Radon space. Proc. 17th Int. Conf. on Medical Image Computing and Computer-Assisted Intervention. Lecture Notes in Computer Science, P. Goland et al. eds: Part I, LNCS 8673, pp. 291298, 2014.

Comparison of Hessian-matrix- and structure-tensor-based methods for myofiber structure extraction from micro-CT volumes H. Oda1, M. Oda1, T. Kitasaka2, T. Akita3, K. Mori1,4 Nagoya University, Graduate School of Information Science, Nagoya, Japan 2 Aichi Institute of Technology, School of Information Science, Toyota, Japan 3 Kanazawa Medical University, Department of Cardiovascular Surgery, Kahoku, Japan 4 Nagoya University, Information & Communications, Nagoya, Japan 1

be computed as the third eigenvector of the HM or the ST. Gaussian functions with the standard deviation rHM or rST were used for computing the HM or the ST, respectively. e3 (p) represents the third eigenvector of the HM or the ST. (2) Fiber tracking of the myofiber (2-1) Defining seed points Seed points on the input volume I were defined in a grid pattern. The minimum distance from one seed point to another was dseed (voxels). The seed points were placed at voxels whose intensities were in the range [tdark, tbright]. The set of seed points has been denoted by S. (2-2) Fiber tracking We performed the fiber tracking of myofiber direction from the seed point. i (0 B i B imax) represents the index of tracking steps. The tracking point at i-th step is written as pi and p-i. At the first step (i = 1) tracking were started from the seed point p0 in S. Fiber tracking was performed for both the direction of the third eigenvector and its opposite, as piþ1 ¼ pi þ wstep e3 ðpi Þ pi1 ¼ pi  wstep e3 ðpi Þ; respectively. wstep is the parameter for the length of one step, and tracking is continued while t B tmax and tdark B i(p) B tbright. If the angle between e3(p) and the previous direction is larger than h, the previous direction is used instead of e3(p). Results We evaluated the outcomes of the HM- and the ST-based methods by applying them to a micro CT volume of the LV of a dog. The volume was obtained by utilizing a micro CT imaging device, inspeXio SMX90CT Plus (Shimadzu, Kyoto, Japan), with 90 kV tube voltage and 250 tube current. The voxel size was 51.6 9 51.6 9 51.6 lm. The volume consisted of 1024 9 1024 9 549 voxels. We used one part of the volume including LV as the input volume, consisted of 924 9 700 9 549 voxels. Figure 1 shows one of the slices of the input volume. Parameters were set as tdark = 2200, tbright=2400, dseed = 30 voxels, rHM = 8 voxels, rST = 8 voxels, imax = 5000, wstep = 1 voxel and h = 60 degrees. Note that we set the same value as rHM and rST for comparison.

Keywords Microstructure  Left ventricle  Visualization  Fiber tracking Purpose The purpose of this paper is to compare results of the Hessian-Matrix(HM) and Structure-Tensor- (ST) based methods for myofiber structure extraction from micro-CT volumes. Myofiber structure extraction method is strongly desired because it is important for the accurate heart beating simulation. ST and fiber tracking have been used in previous method [1]. However, in this study the HM was used instead of the ST, and the outcomes were compared. To the best of our knowledge, although DTI-based fiber tracking has been widely investigated, there are limited reports on CT-based tracking technique. Methods (1) Calculation of myofiber direction Myofiber direction was obtained by tracking the eigenvector direction corresponding to the smallest eigenvalue (third eigenvector) of the HM or the ST, which could be computed at an arbitrary point p on the input volume I. Starting points of tracking were placed in the region of interests. It was hypothesized that the direction with the least intensity gradient would represent the myofiber direction, which can

123

Fig. 1 A slice of the input volume used for the experiment Figure 2a, c show the tracking results obtained by the HM-based methods. Each trajectory had unique shape and direction. It could be

Int J CARS thought that the result reflected the local structure strongly. Therefore, the HM-based method might be useful for obtaining the local myofiber structure, but not for understanding the global myofiber direction.

2

Institut Mines-Te´le´com, Te´le´com Bretagne, Lab-STICC UMR CNRS 6285 E´quipe DECIDE, De´partement Image et Traitement de l’Information, Brest, France 3 Universidad Nacional de Colombia, MindLab Research Group, Bogota´, Colombia Keywords Image retrieval  Alzheimer disease  Magnetic resonance imaging  Image features

Fig. 2 Results (color shows the direction): (a) Result of HM-based method on a slice, (b) Result of ST-based method on a slice, (c) 3D view of the result of HM based-method, (d) 3D view of the result of ST-based method In contrast, the result of the ST-based method shown in Fig. 2b, d was smooth and stable. Neighboring tracked-trajectories showed similar trajectories from the shape and direction perspective. The tracking results could be differentiate between the endocardium, the myocardium and the epicardium regions of an LV region. Although the ST-based method is useful for extracting the myofiber structure macroscopically, some information of local structure such as collagen, fat tissue, cannot be obtained. Conclusion We compared the HM and the ST for fiber tracking of myofiber on a micro CT volume. We clarified that the HM-based method is useful for obtaining the local myofiber structure, while the ST-based method is useful for obtaining the structure macroscopically. Our future work includes quantitative evaluation of the results and pioneer the applications. References [1] Aslanidi OV, et al. (2013) Application of Micro-Computed Tomography With Iodine Staining to Cardiac Imaging, Segmentation, and Computational Model Development, IEEE TMI, Vol. 32, No. 1, pp. 8—17.

Empirical evaluation of general-purpose image features for pathology-oriented image retrieval of Alzheimer Disease cases 1,2

2

3

R. Mendoza-Leo´n , J. Puentes , F.A. Gonza´lez , M. Herna´ndez Hoyos1 1 Universidad de los Andes, Systems and Computing Engineering Department, School of Engineering, Bogota´, Colombia

Purpose To evaluate the applicability of general-purpose image feature algorithms as effective and efficient descriptors for pathology-based image retrieval of Magnetic Resonance Imaging (MRI) brain volumes under the hypothesis that localized quantitative visual similarity associates consistently with the pathological state in Alzheimer Disease (AD). Methods In order to evaluate general-purpose image feature algorithms on the task of retrieving similar subjects according to their demented state, 174 MRI T1 previously pre-processed brain volumes were selected from OASIS dataset [1] (http://www.oasis-brains.org/), comprising 87 healthy and 87 AD subjects. Pre-processing steps included: gain-field normalization, skull removal, an atlas-based registration to the Talairach and Tournoux atlas, and removal of background information. For evaluation purposes, the criteria used to label a case as healthy was a Clinical Dementia Rating (CDR) value of zero (CDR = 0), and greater than zero for demented AD subjects (CDR [ 0). CDR values were also included as part of OASIS data. The CDR is a dementiastaging rating considering six domains for cognition impairment: memory, orientation, problem solving, function in community, home, and personal care. The distribution of selected subjects, according to their CDR was: 87 subjects with CDR = 0 (healthy), 60 subjects with CDR = 0.5 (very mild AD), and 27 subjects with CDR C 1.0 (mild to severe AD). Subjects’ age spanned between 18 and 96 years old, with 44 subjects in the range between 18 and 35 years old, 44 subjects between 35 and 70 years old, 43 subjects between 71 and 79 years old, and 43 subjects between 80 and 96 years old. We selected for evaluation 24 general-purpose image feature descriptors’ algorithms (see Fig. 1). Four different functions were used to measure the similarity between image features of the same kind: L1 distance, L2 distance, Jansen-Shannon Divergence (JSD), and Tanimoto coefficient.

Fig. 1 Averaged MAP performance along the slices on the coronal plane for the 24 evaluated features. Each bar represents a descriptorsimilarity function pair. Only the result for the best similarity function on each descriptor is shown Image groups corresponding to axial, coronal, and sagittal anatomical planes were extracted from the selected volumes and further divided into sets corresponding to the same slice position in all 174 subjects (155 sets for the axial plane, 171 sets in the coronal plane, and 147 sets in the sagittal plane). Then, retrieval performance was

123

Int J CARS evaluated at each set using the Mean Average Precision (MAP) measure independently for each feature and metric combination, totaling 96 MAP evaluations for each set (24 features by 4 similarity measures). The MAP value is the mean of the Average Precision (AP) of query results, obtained by picking each image in the set as query and ranking the remaining images (examples) following a decreasing query-example feature pair similarity (score) order. Accumulative precision and recall values required to evaluate AP are computed based on ranking order and labels (healthy or demented) assigned to each slice according to the case to which the slice belongs. Results In order to identify outperforming descriptions, individual MAP values at each slice in a given anatomical plane were averaged. Figure 1, shows the results of the averaged MAP along the coronal plane for the 24 descriptions evaluated. Feature performance was diverse, ranging from cases with almost random-like MAP behavior (expected random MAP is 0.51), as in the case of the Compactness, Extremal Points, Centroid Feature, and Luminance Layout descriptors, up to averaged MAP values above 0.7 in the case of the Moments and Fuzzy Color Histogram descriptions. Furthermore, three outperforming features were detected in all anatomical planes when analyzed in a slice-by-slice fashion: Auto Color Correlogram, Fuzzy Color Histogram, and Moments. The best MAP values were located at: the 106th axial slice, the 134th coronal slice, and the 129th sagittal slice, being the Fuzzy Color Histogram feature with Tanimoto coefficient the best at all locations (MAP values of 0.763, 0.770, and 0.783, respectively). Additionally, even though the MAP was evaluated independently at each slice set, we also noted a highly similar and continuous pattern of MAP behavior for outperforming descriptors along each anatomical plane. An example of this behavior can be seen in Fig. 2, where also near symmetric pattern is observed along the sagittal plane, with a noticeable decrease of MAP for slices at the medial line (the plane that separates both brain hemispheres, between slices 81 and 94, approximately), which is very consistent with brain morphology and symmetry.

Fig. 2 MAP performance and mean value (in black) along slices on the sagittal plane for three outperforming features. Only the result for the best metric on each descriptor is shown Conclusion Obtained results strongly suggest that quantitative visual similarity, at certain anatomical locations, appears to associate consistently with the pathological state. Specifically, when analyzing the different anatomical planes, some specific locations of the brain MRI volumes seemed to contain visual aspects more closely related with healthy and demented cases than others. Additionally, based on the results of this extensive evaluation of general-purpose image features, this work also suggests that despite their simplicity and general-purpose nature, quantitative features may allow to retrieve pathologically similar MRI images. Interpretation of the results may also be complemented with the ordered relevance rankings that are returned, which may provide further clinically-meaningful information during AD identification in

123

new subjects. Finally, retrieval performance results illustrate that general-purpose image features may also be useful for other pathology-oriented retrieval tasks in brain images, with the added benefit of being simple and efficient (evaluation of one 2D slice is only needed), and thus probably suitable for large scale systems. Acknowledgments The authors wish to thank Fundacio´n CEIBA and Alcaldı´a Mayor de Bogota´, for the financial support of Ricardo Mendoza’s PhD studies through the scholarship program ‘‘Becas Rodolfo Llina´s’’, and Amazon Inc., for providing valuable computing resources through an ‘‘AWS in Education Research’’ grant. References [1] Marcus D, Wang T, Parker J, Csernansky J, Morris J, and Buckner R (2007) Open Access Series of Imaging Studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. Journal of cognitive neuroscience, vol. 19, no. 9, pp. 1498–1507.

Chest imaging platform: an open-source library and workstation for quantitative chest imaging J. Onieva1, J. Ross1, R. Harmouche1, A. Yarmarkovich1, J. Lee1, A. Dı´az1, G. R. Washko1, R. San Jose´ Este´par1 1 Brigham and Women’s Hospital, Radiology, Boston, United States Keywords Medical workstations  Open source tools  COPD/Cancer biomarkers  CT processing Purpose Chronic obstructive pulmonary disease (COPD) and lung cancer have one of the highest mortality rates in chronic diseases and cancer. Image-based phenotyping of chest diseases has become of paramount importance to better understand, characterize, diagnose and treat these conditions. Therefore, there is a need for open-source toolkits that can enable the implementation of new algorithms as well as the computation of quantitative phenotypes that can be used in clinical settings. Methods We have developed an open-source library, the Chest Imaging Platform (CIP: www.chestimagingplatform.org) that implements the computational components needed for the extraction of CT-based phenotypes, mainly anatomy segmentation and phenotype quantification. We also provide a user-friendly workstation for the analysis of clinical CT based on 3D Slicer. The library is built on top of known open-source general-purpose libraries for visualization and image analysis, namely, VTK (www.vtk.org), ITK (www.itk.org), Teem (teem.sourceforge.net). The library is written in both C ++ and python and provides both core classes as well as command line tools that form the basis of the computational components. The library design is such that allows for extensibility so other groups and developers can contribute to the project. Figure 1A shows the CIP architecture and its dependencies. In addition to that, SlicerCIP is a 3D Slicer extension that leverages CIP to provide a workstation environment with clinically oriented workflows. 3D Slicer is a general-purpose research radiology workstation with a modular architecture. SlicerCIP provides GUI-based access to the command line tools in CIP as well as dedicated modules that integrate user interactions and automatic approaches in an interactive environment for disease phenotype extraction. SlicerCIP also provides end-to-end workflows that integrate the CLI and module functionality to provide patient-oriented quantification for clinicians and specialist. Figure 1B shows the SlicerCIP architecture and its interaction with CIP. Software quality assurance is supported by means of version control (git) and nightly continuous building and testing using CDash [1–2].

Int J CARS Conclusion The CIP and SlicerCIP are the evolution of the Airway Inspector project [3] and provide an extensible, open-source environment that facilitates chest-disease research by technical developers and clinical scientists. References [1] http://cdash.airwayinspector.org/index.php?project=CIP [2] http://www.airwayinspector.org

Fig. 1 CIP and SlicerCIP architectures (A) and detailed view on the CIP library architectures (B) Results CIP currently has 457,053 lines of code in 1,312 files and 66 command line tools that encapsulate the basic functionality for lung mask extraction, lobe segmentation, particles processing and lung registration. CIP also provides 31 python classes and 18 python command line tools for feature (airway, vessel and fissure) sampling based on scale-space particles, pectoralis muscle segmentation, and deformable registration for registration-based gas trapping phenotypes. The python component also provides a phenotype extraction infrastructure to facilitate the computation of phenotypes for different anatomical regions. SlicerCIP currently implements a series of clinical modules to perform automatic lung segmentation, interactive lobe segmentation, emphysema quantification (see Fig. 2A), semi-automatic body composition analysis and tumor volume quantification (see Fig. 2B). It also provides visualization of CIP region-type labelmaps and scale-space particles. SlicerCIP is distributed as an Slicer extension and it can be downloaded directly from the Slicer Extension Manager.

Fig. 2 Module for the quantification of emphysema and gas trapping by lobes (A) and module for the segmentation and assessment of pulmonary nodules (B)

Three dimensional data generation and graphical representation of theoretical tracheobronchial trees and lung models C. Ciobirca1, G. Gruionu2, T. Lango3, H. Olav Leira4, S. D. Pastrama1, L. G. Gruionu5, T. Popa5 1 University Politehnica, Department of Strength of Materials, Bucharest, Romania 2 Harvard University, Edwin L. Steele Laboratory for Tumor Biology, Boston, USA 3 SINTEF Technology and Society, Department of Medical Technology, Trondheim, Norway 4 St. Olavs Hospital, Department of Thoracic Medicine, Trondheim, Norway 5 University of Craiova, Department of Mechanics, Craiova, Romania Keywords Virtual bronchoscopy  Lungs model  Scalar fields  Algorithms testing Purpose Developing Image Guided Therapy (IGT) software applications for bronchoscopy requires the implementation of various sophisticated algorithms, such as segmentation of human lungs from the CT scans, virtual navigation and procedure planning, tracking of medical instruments during the real procedure, etc. [1,2]. These algorithms need ‘‘simple’’ data (simpler than the human CT scans) to be tested on during the implementation phase and also for the unit testing phase. Starting from a theoretical model of the human tracheobronchial tree, we currently implement a method that generates three dimensional data sets similar to the CT scans, that can be represented as volumes using volumetric representation algorithms from VTK. Also, we apply on these data sets a VTK pipeline based on the Marching Cubes algorithm and extract the surface of the ‘‘airways’’ [3]. We have used this generator to develop and test a collision and detection algorithm (used for virtual navigation in virtual bronchoscopy procedures), test a segmentation algorithm implemented in CustusX platform, develop and test a rigid registration method. Methods The starting point for our generator is the Weibel’s theoretical model of the human tracheobronchial tree [4]. In this model, the trachea is the first branch (level zero) and, at each level, each parent branch generates exactly two children branches. Such that, at level k there are 2 k branches. An intersection point has three branches (the parent and its two children). We consider the branches as straight lines, but the generator is not limited to this consideration: it could take into account branches with more complicated shapes. At each level, the model prescribes the diameter and the length of the airways. These values can be found in references. Starting from the model, we generate a scalar field with the symmetry of the tracheobronchial tree. A scalar field is a function that associates a scalar value to a point from a region of the three dimensional space, f : R3 ? R. By the symmetry of the tracheobronchial tree, we mean that in fact the scalar value depends on the distance from the point to the nearest branch of the tree, f(x,y,z) = f(rnearest_branch, …). In the previous formula, ‘‘…’’ represents supplementary information that describes the geometrical placement of the point around the branch (this information can be

123

Int J CARS used to alter the symmetry: the tube diameter can change along its length, the cross section can be deformed from circular to various shapes, etc.). We implemented these prescriptions in two C++ classes, VolumeGenerator and VolumeSampleFunction (based on vtkImplicitFunction). An instance of vtkImageData is generated at the output port of a vtkSampleFunction instance (which uses our class VolumeSampleFunction). The vtkImagedata instance is a collection of voxels in a cubic region of the three dimensional space that are rendered using volumetric methods from VTK framework, with a transfer function that marks three regions: air inside the tubes, lungs tissue, boundary (Fig. 1). On the generated data set, we apply the following VTK pipeline: vtkImageData ? vtkImageShrink3D ? vtkGaussianSmooth ? vtkMarchingCubes, and render the ‘‘airways’’ surface obtained at the output port (Fig. 2).

Fig. 1 Representation of the generated scalar field on a cubic region, divided in 512 x 512 x 256 voxels, 0.5 mm 9 0.5 mm 9 1 mm each voxel (3D representation, 2D axial, sagittal, coronal planes)

surface and a tool representation (either a medical tool, or a virtual VTK camera) is constrained to remain always inside this surface. Also, we used our generator data sets to test a segmentation method (‘‘Tubes Segmentation Framework’’) implemented in the CustusX platform, developed at SINTEF, Trondheim, Norway. We are developing a rigid registration method for electromagnetic tracking based on a fiducial point (a reference sensor placed on the patient) and a constraint surface. During development and initial testing of this method, we use simple data sets from our generator. Conclusion Simple data sets that resemble human tracheobronchial trees and lungs are an important helping tool in the development of sophisticated algorithms for bronchoscopy IGT applications. References [1] Smistad E, Falch T L, Bozorgi M, Elster A C, Lindseth F (2015) Medical image segmentation on GPUs—A comprehensive review. Medical Image Analysis 20(1): 1–18. [2] Smistad E, Elster A C, Lindseth F (2013) GPU accelerated segmentation and centerline extraction of tubular structures from medical images. International Journal of Computer Assisted Radiology and Surgery. 9(4): 561–575. [3] Ciobirca C, Popa T, Gruionu G, Lango T, Leira H O, Pastrama S D, Gruionu L G (2016) Virtual bronchoscopy method based on marching cubes and an efficient collision detection and resolution algorithm. Cieˆncia & Tecnologia dos Materiais. In print (accepted 15.12.2015). [4] Weibel E (1963) Morphometry of the Human Lung, Academic Press, Inc., New York. Acknowledgement The research leading to these results has received funding from EEA Financial Mechanism 2009–2014 under the project EEA-JRP-RONO-2013-1-0123—Navigation System for Confocal Laser Endomicroscopy to Improve Optical Biopsy of Peripheral Lesions in the Lungs (NAVICAD), contract no. 3SEE/30.06.2014.

Estimation of renal vascular dominant regions using Voronoi diagram C. Wang1, M. Kagajo1, Y. Nakamura2, M. Oda1, Y. Yoshino3, T. Yamamoto3, K. Mori1 1 Nagoya University, Graduate School of Information Science, Nagoya, Japan 2 Tomakomai College, National Institute of Technology, Tomakomai, Japan 3 Nagoya University, Graduate School of Medicine, Naogya, Japan Keywords Segmentation  Graph-cut  Template model  Voronoi diagram

Fig. 2 ‘‘Airways’’ surface obtained from Marching Cubes algorithm applied on the region from Fig. 1 Results We used these ‘‘theoretical’’ data sets to implement a collision detection and resolution algorithm used for virtual navigation and tracking corrections in bronchoscopy procedures. The airways tubes surface resulted from the Marching Cubes algorithm is the collisions

123

Purpose Nowadays, partial nephrectomy has been a common treatment for kidney cancer, which can maintain a high residual renal function in patients. Blood vessel clamping is a critical issue in the partial nephrectomy. In this work, we make a contribution to provide a better and accurate renal artery segmentation approach utilizing graph-cut and template model tracking methods. We estimate the dominant region of each renal artery which would facilitate identifying the blood vessels which feed the tumor. Physicians can easily make a surgical plan using these diagnosis information to decide the blood vessel clamping scheme. Methods We utilized the graph-cut [1] method to extract the thick blood vessels from an input image data. In this work, tubular enhancement filter is utilized to extract the tubular structures as foreground for the graphcut, and k-means clustering method is then applied to extract the most

Int J CARS probable blood vessel regions. The highest intensity component is chosen as blood vessel regions as foreground, and take the rest as possible background regions. After that, the graph-cut method is performed to extract the rough segmentation of the blood vessels. Finally, a reasonable accurate vessel segmentation result is obtained. Template model tracking was utilized presented by Friman et al. [2] to extract tiny tubular structures. Template model tracking will start at the end points of each thick branch extracted by the rough segmentation and continue to extract the tiny blood vessels. A local image model fitting is used to minimize the difference between image model and the input image data. Thus a least-squares method is utilized to solve the minimization problem. As for tracking procedure, after the first model was created at the end point of each blood vessels in the rough segmentation data, several candidate vessel models are generated. The best fitted model is chosen as the next definite vessel model. The tracking will continue until the estimation score, computed by Student’s t-Test, satisfies the terminal condition. To suppress the over-segmentation caused by template model tracking, a novel method is proposed to discriminate the probable over-segmentation using local geometric features. Local geometric features are obtained by performing raycasting in a local hemisphere. We estimate renal vascular dominant regions utilizing the Voronoi diagram. Each branch of renal arteries is considered as a set of ‘‘seed’’ points of the Voronoi diagram instead of using the end points of arteries. Results We evaluated the proposed segmentation method on 8 cases of CT volume data. Experiments were performed on VOI of kidney which are cubic image data containing the whole kidney region created manually. All the algorithms are implemented in C ++. All cases are processed on a normal computer with a Intel(R) Xeon(R) CPU E52667 2.9 GHz 6 cores. Gold standards were created by two human experts with medical experience. In this paper, Dice index, False Positive Rate (FPR) and True Positive Rate (TPR) were utilized to validate the proposed method. Detailed experimental results are shown in Table 1. It is obvious that the proposed method is more accurate than the previous method [3]. Over-segmentation caused by template model tracking is suppressed by hemi-sphere geometric restriction effectively. An experimental result on 3-D CT volume is shown in Fig. 1.

Table 1 Experimental results. For each cases, results of proposed method are shown at left column, and results of the previous work [3] are shown at right column Case 1

Case 2

Case 3

Case 4

Dice(%) 76.02 62.35 75.24 46.9

85.89 77.62 64.19 62.84

TPR(%) 81.26 69.68 82.4

87.12 72.22 51.61 49.6

FPR(%)

0.14

0.23

Case 5

0.11

74.8 0.41

Case 6

Dice(%) 77.64 66.11 90.1

0.11

0.1

Case 7

0.07

0.07

Case 8

53.58 89.79 45.29 95.99 68.59

TPR(%) 99.05 82.12 92.02 53.99 88.15 60.99 97.54 60.34 FPR(%)

0.31

0.37

0.03

0.12

0.02

0.36

0.02

0.06

Fig. 1 Experimental result of hemisphere surface restriction. Left) There is serious over-segmentation leaking into renal vein region. Center) Over-segmentation is prevented successfully by hemisphere surface restriction. Right) A gold standard created by human experts Conclusion The main contribution of this work is to propose a novel method using graph-cut and template model tracking to segment the renal arteries of a highly accuracy, and estimate the renal vascular dominant regions successfully utilizing the proposed vessel segmentation method. The experimental results showed that the proposed method is more accurate than the previous method. Thus the estimation of vascular dominant region is more reliable. The proposed framework including renal artery segmentation and estimation of vascular dominant region could provide more useful information for partial nephrectomy surgery. References [1] Boykov, Yuri Y, Jolly, MP (2001) Interactive graph cuts for optimal boundary & region segmentation of objects in ND images. In Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on, vol. 1, pp. 105–112. IEEE, 2001. [2] Friman, O, Hindennach M, Ku¨hnel C, Peitgen HO (2010) Multiple hypothesis template tracking of small 3D vessel structures Medical image analysis 14, no. 2 (2010): 160–171. [3] Oda, M, Kagajo M, Yamamoto T, Yoshino Y, Mori K (2015) Size-by-size iterative segmentation method of blood vessels from CT volumes and its application method of blood vessels from CT volumes and its application to renal vasculature, International Journal of Computer Assisted Radiology and Surgery, Vol. 10, Sup. 1, pp. S208-S210, 2015.

MRI Minkowski functionals as a prognostic indicator for breast cancer M. Fox1, P. Gibbs1, M. Pickles1 1 Hull York Medical School, CMRI, Hull, Great Britain Keywords Texture analysis  Breast  MRI  Triple negative breast cancer Purpose Minkowski Functionals (MF) have been used as a method of texture analysis in CT for many years [1]. Early work suggests that they may provide a new way of observing and describing tumours in MRI, and may be useful as a diagnostic/prognostic tool, or in response prediction [2,3]. This work investigates whether MFs can be used to distinguish between triple negative breast cancer (TNBC) and other tumor intrinsic subtypes.

123

Int J CARS Methods MR scans were taken before neo-adjuvant chemotherapy began of 183 patients using a dedicated breast coil on a 3T HDx scanner (GE Healthcare, Milwaukee, WI), all with biopsy confirmed breast cancer and with a median age of 49 years (21–70 years). A fat nulled, Volume Imaging Breast Assessment (VIBRANT) sequence was used to produce sagittal, T1 weighted images. Software was developed in house using MatLab to segment tumors from scans, and threshold the resulting cropped images to varying threshold levels ranging from 10 to 100. Analysis was carried out on images taken before contrast agent was added, and at 1 - 5 min post contrast For each thresholded image (examples in Fig. 1.) three MF values were perimeter (U = calculated representing area (A = ns), -4 ns + 2ne), and Euler value (v = ns- ne + nv), where ns = number of white pixels, ne = number of edges, and nv = number of vertices; MF values were then standardized to account for tumor size differences. Analysis was conducted by calculating 6th order polynomials for each patient describing the change in each MF value as the threshold was raised. Binary logistic regression models were created on the train set (n = 100) and validated on the test set (n = 83) using ROC curves to analyze model performance. Patients were split into groups based on tumor intrinsic subtype, triple negative breast cancer (TNBC) and all other intrinsic types.

Fig. 1 A comparison between triple negative (TNBC) and all other (Other) intrinsic subtype breast cancer tumors as analyzed using Minkowski Functionals. Both tumors are grade III, of no specific type (NST), images are taken from 1-min post-contrast and are grouped into columns of identical threshold levels for direct comparison Results AUC values above 0.800 were consistently found, and corroborated, for all step sizes at varying time points. The threshold step size which performed the most consistently was found to be 30 threshold steps, with all post-contrast time points returning significant ([ 0.750) AUC values with the best performance being 0.915 (0.843–0.970 95 % CI) at 3-min post-contrast. The largest AUC value overall was 0.935 (0.869–1.000 95 % CI) for 1 min post-contrast, using 10 threshold steps, however no other results were returned at 10 threshold steps. Conclusion In this work we show that Minkowski Functionals may be used to differentiate triple negative breast cancer from cancers of other intrinsic subtypes. We also suggest that using too many threshold steps when calculating MFs will results in reduced performance, agreeing with previous findings [4]. These findings demonstrate the potential utility of MF in the diagnosis/treatment planning of breast cancer patients. References [1] Boehm H, et al. Automated classification of normal and pathologic pulmonary tissue by topological texture features extracted rom multi detector CT in 3D. EUR RADIOL, 2008. [2] Larkin TJ, et al. Analysis of Image Heterogeneity Using 2D Minkowski Functionals Detects Tumor Responses to Treatment. Magnet Reson Med, 2014. [3] Canuto HC, et al. Characterization of Image Heterogeneity Using 2D Minkowski Functionals Increases the Sensitivity of Detection of a Targeted MRI Contrast Agent. Magnet Reson Med, 2009.

123

[4]

Nagarajan MB, et al. Classification of small lesions in dynamic breast MRI: eliminating the need for precise lesion segmentation through spatio-temporal analysis of contrast enhancement. Mach Vision Appl, 2013.

Segmentation overlap measures are biased to structure’s size but correctable R. R. Shamir1, Y. Duchin1,2, J. Kim3, G. Sapiro3, N. Harel2 1 Surgical Information Sciences, Minneapolis, United States 2 University of Minnesota, Department of Radiology, Minneapolis, United States 3 Duke University, Departments of Electrical & Computer Engineering, CS, and BME, Durham, United States Keywords Segmentation  Validation  Volume overlap  Dice coefficient Purpose Overlap measures such as dice coefficient (DC) are extensively reported in studies evaluating segmentation algorithms. However, these measures heavily depend on the volume of the segmented structure and image resolution, among other factors [1]. We first demonstrate this dependency for a hypothetical cube shaped structure for which we suggest a correction factor. Then, we suggest a method to normalize the DC and test it for two common neurosurgical targets: the subthalamic nucleus (STN) and the thalamus. Methods Rough theoretical estimate of the dice coefficient bias—As a first step to demonstrate the dependence of the dice coefficient (DC) on structure’s size, let’s assume that the structure’s shape is a cube with edge length of k and that the segmentation is the actual structure translated by t along one of the cube edges (Fig. 1). In this case the dice coefficient is 2(k2(k-t))/2 k3 that is equal to (1) (k-t)/k.

Fig. 1 Overlap measure is highly related to segmented structure size. In this simplified example we simulate a translation segmentation error of a cube-shaped structure. In this case, the volume of the overlap (marked with grey) is (k-t)k2. In this case, the maximal value of the dice coefficient is (k-t)/k

Int J CARS Where t \ k; otherwise the DC is zero. The translation error t can be chosen to account for the voxel’s size, model intra- or inter-observer segmentation accuracy, and/or incorporate the distances between expert’s and computed segmentations centers of mass. In practice, image quality is limited. Therefore, t [ 0 and the DC can never reach its maximal theoretical value. Moreover, Equation 1 suggests that smaller structures are associated with lower DC values and supports previous studies [1]. Readers that do not have direct access to data can use this rough estimate to evaluate the DC values reported in studies on segmentation methods. We provide an example in the results section. Next, we present a more accurate numerical method to estimate a correction factor for the DC value. Numerical computation of dice coefficient bias—We suggest the following method to estimate a correction factor to the DC value that accounts for the structure’s size: Volume-correction DC factor a) Iteratively repeat: 1) Draw a random translation vector to model expected errors. May incorporate • • •

a. Voxel size b. Inter/intra observer variability c. Centers of mass distances

2) Translate a copy of the segmented structure 3) Compute the DC between translated and original segmented structures b) Compute the mean value of all the computed DC values. Note that we modeled only a translation error. In practice, the segmentation errors may incorporate additional transformations or deformations. Therefore, this simulation is conservative. The resulted average DC value is most likely higher than the actual possible average DC value and represents an upper bound. In this study we used two models for the translation error vector. In the first we draw a random vector from normal zero-mean distribution and with variance equals to the voxels’ size. For the second model we automatically computed the segmentation of 14 STNs of 7 Parkinson’s disease patients on clinical T2 MR images (voxel size of 0.5mm 9 0.5mm 9 2 mm) as described in [2]. The average difference between experts and computed STNs’ center of mass was -0.05, 0.64 and -0.3 mm along the x, y, and z axes respectively. The standard deviation was 0.5, 0.5 and 1 mm along the same axes. We used these values under independent normal distribution to model the second translation error. Results Rough theoretical estimate—The thalamus and the STN are associated with volumes of about 7200 mm3 and 150 mm3, respectively. We estimated k as the cube root of the volumes and received k = 19.3 mm and 5.3 mm for thalamus and STN, respectively. We incorporate images with voxel size of 0.5mmx0.5mmx2 mm, and model the translation error size as t = |(0.5,0.5,2)| = 2.12 mm. As a result we get DC correction factors of 0.60 and 0.89 for the STN and thalamus, respectively. Numerical simulation results—We implemented and ran the simulation above for the STN and thalamus that were segmented manually by experts on high-field (7T) MRI [3]. When the translation error variance was modeled with voxels’ size, the DC correction factor was estimated at 0.68 and 0.93 for STN and thalamus, respectively. Modeling the translation error with the center of mass differences distribution, the DC correction factor was estimated at 0.64 and 0.88 for STN and thalamus, respectively. Automatic segmentation results—The average DC on the 14 automatically segmented STNs was 0.6 (SD = 0.13). The average distance between the segmented surfaces was 0.8 mm (SD = 0.2 mm). Therefore, the average DC of 0.6 seems to be associated with excellent segmentation result. Figure 2 presents a specific example of low DC, but accurate segmentation.

Fig. 2 An example for low DC on a small structure with good segmentation result. Ground truth (green) and automatically computed (red) subthalamic nucleus (STN). Although the segmentation is highly accurate in terms of Euclidian distance (1.3 mm between centers of mass and similar error on the surface) the DC in this example is 0.27 Conclusion Volume overlap measures such as DC are extremely sensitive to the structure’s volume. DC values of 0.6 were associated with submillimeter accuracy. Therefore, the DC is not a suitable index for evaluating accuracy of segmentation for small structures and need to be corrected. Readers of segmentation studies that do not have direct access to data can apply Equation 1 to roughly correct reported DC values and compare results for various structures on a normalized scale. Researchers can estimate the DC correction factor more accurately with the iterative simulation method presented above. References [1] Rohlfing T (2012) Image similarity and tissue overlaps as surrogates for image registration accuracy: widely used but unreliable. IEEE Trans Med Imaging 31:153–63. [2] Kim J, Duchin Y, Sapiro G, et al. (2015) Clinical deep brain stimulation region prediction using regression forests from highfield MRI. In: 2015 IEEE Int. Conf. Image Process. IEEE, pp 2480–2484. [3] Lenglet C, Abosch A, Yacoub E, et al. (2012) Comprehensive in vivo mapping of the human basal ganglia and thalamic connectome in individuals using 7T MRI. PLoS One 7:e29153.

Computer assisted pathological image segmentation using Markov random field T. Mungle1, S. Tewary1, D. Das1, I. Arun2, R. Ahmed2, S. Chaterjee2, A. Maiti3, C. Chakraborty1 1 IIT Kharagpur, School of medical science and Technology, Kharagpur, India 2 Tata medical centre, Kolkata, India 3 Midnapur medical college and hospital, Midnapur, India Keywords Pathology images  Segmentation  Markov random field  Optimization techniques

123

Int J CARS Purpose In the field of digital pathology, researchers have been aiming to develop medical image analytic software for assisting pathologists and subsequently computer-aided diagnosis (CAD). One of the significant tasks in designing a CAD system is medical image segmentation for delineating the expected region of interest. In past few decades, several image segmentation techniques have been evolved depending upon the application. Pathology image analysis is employed for various cancer screening, blood-related disorders including malaria, dengue, leukaemia etc. Recognition of abnormal signatures in pathological slides is a cumbersome task for a pathologist rapidly and efficiently. Segmentation is one of the efficient ways to automatically characterize such abnormal regions/signatures [1]. In view of this developing an efficient segmentation algorithm for a CAD system is essential to provide improved and fast treatment facilities. Methods In view of this, we propose a Markov Random Field (MRF) based segmentation to obtain region of interest, which can be used later for decision support systems. Initially, digital images obtained from pathology slides yields color differences and slides from different labs add to the problem. White balancing is incorporated to overcome the colorimetric differences followed by normalization [2]. MRF modelling [3] is employed to obtain essential cells/nuclei for analysis. K-means clustering is used for initial labelling as a precursor for MRF model. Further, we use optimization techniques [4, 5] (Expectation Maximization (EM), Iterated Conditional Modes (ICM), and Gibbs Sampler) in association with MRF to obtain more accurate segmentation results. Results The results for segmentation are given in Fig. 1 for various pathological images. It shows output of all three optimization techniques along appropriate ground-truth. The ground observations assisted us in the preliminary visual validation of output segmented data in the form of comparisons. It is observed that with Gibbs sampler optimization there is a little information loss in terms of some region of interest not segmented. The execution time for the three optimization techniques is approximately 30, 12 and 28 s respectively given the methods are iterated over 5 loops (on Intel Core i5-2400 CPU @ 3.10 GHz and 4 GB RAM). The parameter b, which defines spatial interaction among neighbouring pixels, is predefined with a value of 2.5 for our work. The parameter is independent of number of iterations performed and as the value increases loss of information can be observed.

Fig. 1 Segmented region of interest from various pathological images Conclusion The developed method has been tested for molecular expression characterization of breast carcinoma histopathological images and nucleus of white blood cells from peripheral blood smear images for leukaemia detection. Proposed segmentation method is evaluated for all three optimization techniques in terms of time complexity. The results from each optimization method are compared visually along

123

with confirmation from expert pathologists where it is found that, in case of Gibbs optimization there is diminutive information loss in segmentation. As far as, time complexity is concerned, it is observed that ICM method is fastest among all to extract region of interest. We propose to use ICM optimization could be employed for development of an efficient CAD system for microscopic pathological image analysis in various disease diagnoses. Acknowledgement We would like to thank Ministry of Human Resource Development (MHRD), Government of India for supporting and financially aiding the study under the research grant, 4-23/2014 -T.S.I. dated: 14-022014. References [1] Gonzalez RC, Woods RE (2008) Digital image processing: Pearson prentice hall. Upper Saddle River, NJ. [2] Banic N, Loncaric S (2014) Improving the White patch method by subsampling. In Image Processing (ICIP), 2014 IEEE International Conference on (pp. 605–609). IEEE. [3] Geman S, Geman D (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. Pattern Analysis and Machine Intelligence, IEEE Transactions on, (6), 721–741. [4] Besag J (1986) On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society. Series B (Methodological), 259–302. [5] Blake A, Kohli P, Rother C (2011) Markov random fields for vision and image processing. MIT Press.

Segmentation method of abdominal arteries from CT volumes utilizing intensity transition along arteries M. Oda1, T. Yamamoto2, Y. Yoshino2, K. Mori1,3 1 Nagoya University, Graduate School of Information Science, Nagoya, Japan 2 Nagoya University, Graduate School of Medicine, Nagoya, Japan 3 Nagoya University, Information and Communications, Nagoya, Japan Keywords Segmentation  Intensity transition  Abdominal artery  CT image Purpose The blood vessels run around and inside many organs. Blood vessel positions and structures information can be used for lesion detection, surgical planning, and computer assisted surgery. Blood vessel segmentation is an essential process to obtain blood vessel information. Many blood vessel segmentation methods from CT volumes have been proposed [1–3]. A multi-scale line structure enhancement filterbased method [3] segments abdominal arteries from thick to thin parts by using an iterative segmentation process. However, this method caused many false positives (FPs) including bone and portal vein regions, which have higher and lower CT values compared to artery regions. We present an abdominal artery segmentation method from CT volumes utilizing over segmentation reduction based on CT value transitions along arteries. CT values of arteries are gradually change along the arteries. The proposed method iteratively track abdominal arteries from thick to thin parts based on line structure enhancement results. In the tracking process, a CT value change along tracked regions is checked to stop false tracking of out of artery regions. Methods 1. Overview The input of our method is an arterial phase contrasted abdominal CT volume. A rough segmentation is performed as the Ref. [3] to segment thick arteries including the abdominal aorta. A precious

Int J CARS segmentation is performed subsequently. The precious segmentation segments thick to thin arteries iteratively by using the line structure enhancement filter in the similar framework as the Ref. [3]. For each iteration step, we newly introduce the FP reduction process that utilizes a CT value transition along tracked regions. This process is described in the next section. The regions obtained in the rough and precious segmentations are output of our segmentation method. 2. FP reduction process utilizing intensity transition along blood vessel This process is used in the iterative segmentation process of abdominal arteries. Details of the iterative segmentation process can be found in the Ref. [3]. The segmentation result in the n-th iteration is described as On. O1 is obtained by the rough segmentation. Connected components in On are described as cn,i (i = 1,…,In). cn,i is thought as a candidate of segmentation result. For each voxel in cn,i, the maximum CT value in a local region is calculated. We define an average of the local maximum CT values among all voxels in cn,i as dn,i. The median point of cn,i is represented as mn,i. From voxels in On-1, a set of voxels en,i that consists of voxel p which satisfies ||p-mn,i|| \ t1 mm. For each voxel in en,i, the maximum CT value in a local region is calculated. We define an average of the local maximum CT values among all voxels in en,i as fn,i. The dn,i and fn,i are averages of the local maximum CT values in cn,i and its surrounding region, which is segmented in the (n-1)-th iteration. If the condition |dn,i-fn,i| [ t2 is satisfied, the connected component cn,i is eliminated from candidates of segmentation result. The remaining connected components in On are regarded as the final segmentation result in the n-th iteration. Results We performed abdominal artery segmentation by using the proposed method to four cases of arterial phase contrasted abdominal CT volumes. We experimentally determined the parameters as t1 = 10 mm and t2 = 30. The precision and recall rates for voxels of segmented regions were 91.3 % and 80.0 %. The precision rate was improved from the previous method [3] (the precision and recall rates of [3] were 89.6 % and 81.2 %). Figure 1 shows a ground truth of a case and Fig. 2 shows segmentation results of the proposed method and the previous method [3] of the case.

Fig. 2 Segmentation results of the proposed method (left) and the previous method [3] (right) In the segmentation results obtained by using the proposed FP reduction process, many arteries were correctly segmented. Especially, FPs of segmentation results were effectively suppressed. The previous method [3] caused many FPs in kidney, portal vein, and bone regions. The artery runs near these organs, and these organs have similar CT values to contrasted artery regions. In the artery segmentation (or tracking) of the previous segmentation processes caused false tracking to these organs. The new FP reduction process in this paper effectively reduced these false trackings. CT value changes along the artery are small in local regions. Thus, we calculate difference of CT values along segmented regions to evaluate CT value changes. Tracking of the artery in the segmentation process is stopped if CT value change is large. It reduces FPs in segmentation results. In the segmentation result, parts of the kidney and the vein were included. These regions have similar CT values and structures as the artery. Reduction of these regions utilizing anatomical knowledge will be necessary. Conclusion This paper proposed a FP reduction process utilizing intensity transition along the blood vessels. The precision and recall rates of the method were 91.3 % and 80.0 %. Future work includes further reduction of FPs utilizing anatomical knowledge and evaluation by using many cases. References [1] Schneider M, Hirsc S, Weber B, et al. (2014) ‘‘Joint 3-D vessel segmentation and centerline extraction using oblique Hough forests with steerable filters, ‘‘ Med Imag Anal, Vol.19, No.1, pp.220–249. [2] Cherry KM, Peplinski B, Kim L, et al. (2014) ‘‘Sequential Monte Carlo tracking of the marginal artery by multiple cue fusion and random forest regression,’’ Med Imag Anal, Vol.19, No.1, pp.164–175. [3] Oda M, Kagajo M, Yamamoto T, Yoshino Y, Mori K (2015) ‘‘Size-by-size iterative segmentation method of blood vessels from CT volumes and its application method of blood vessels from CT volumes and its application to renal vasculature,’’ Int J CARS, Vol.10, Sup.1, pp.S208–S210.

Fig. 1 A ground truth of a case

123

Int J CARS Differentiation of carcinomas from bladder wall for cancer staging: a preliminary study X. Xu1, S. Yu2, Y. Liu1, G. Zhang1, D. Xiao1, X. Zhang1, Q. Tian3, H. Lu1 1 Fourth Military Medical University, Biomedical engineering, xi’an, China 2 Chinese Academy of Sciences,Shenzhen Institutes of Advanced Technology, Shenzhen Key Laboratory for Low-Cost Healthcare, Shenzhen, China 3 Fourth Military Medical University, Tangdu Hospital, Department of Radiology, xi’an, China Keywords Bladder cancer  Invasion depth  Cancer staging  GrabCut Purpose Bladder cancer is a severe common tumor that threatens human health worldwide [1]. The invasion depth of carcinomatous tissue penetrating bladder wall, namely the staging of cancer, is critical for the diagnosis, treatment and management of this disease. Considering the non-invasive nature and superior capability in evaluating perivesical infiltration of bladder tumors of magnetic resonance (MR) imaging, in this study, a preliminary scheme is proposed to differentiate bladder carcinomas from wall tissues for staging prognosis using MR images. Methods The whole scheme consists of four steps: (1) MR image acquisition, (2) bladder wall segmentation via GrabCut, (3) tumor extraction, and (4) staging prognosis. Detailed description of the proposed scheme is illustrated below. MR image acquisition:A 3D axial cube T2-weighted sequence was used for high-resolution image acquisition. Each subject was scanned by a clinical whole body scanner (GE Discovery MR750 3.0T) with a phased array body coil. They were examined in supine position without contrast agent injected. Preliminarily, 10 MR datasets of patients with bladder cancer were involved and each contained over 100 image slices. Meanwhile, the histopathological results were obtained from the postoperative biopsy. Bladder wall segmentation via GrabCut:GrabCut is an iterative graph cuts method that extracts an object from its background [2, 3]. For each slice of bladder images, the whole bladder wall, including both tumor and wall tissues, was segmented as shown in Fig. 1 (a). In total, we obtained 10 datasets of segmented bladder wall images, and the average image slices of each dataset was 86.

Tumor extraction: With the entire bladder wall segmented, GrabCut was utilized again to further extract tumors and differentiate them from wall tissues. As shown in Fig. 1 (b), the extraction result actually reflects the invasion depth of tumors into the wall tissues and subsequently suggests the cancer staging. Clinical evaluation: For quantitative prognosis of cancer staging, the relative invasion depth Rwt can be calculated based on the ratio of wtmin and WTave, where wtmin denotes the minimum wall thickness of wall tissues left in the tumor region (the red rectangle region in Fig. 1 (c)) after the tumor removal; WTave represents the average thickness of the bladder wall out of the tumor region. According to our previous study, the isopotential lines can be depicted on the potential field of the bladder wall, which may reflect the pathological changes of bladder cancer [4]. Therefore, the relationship between and staging can be preliminarily listed as follows: if Rwt C 0.9, the staging may be Tis or T1; if 0.5 B Rwt \ 0.9, the staging may be T2a; if 0 \ Rwt \ 0.5, the staging may be T2b; if Rwt = 0, it suggests that the tumor penetrates the bladder wall and the staging may be T3. Results For the segmentation of the bladder wall, the results using GrabCut were compared with that using coupled directional level-set (CDLS), as shown in Fig. 2. Compared with the bladder wall outlined manually by experienced radiologists as the groundtruth, the mean accuracy of the segmentation via CDLS and GrabCut is 94.5 % and 96.5 %, respectively. Besides, the mean time consumption of the segmenting is 30 and 5 s per slice. Preliminary staging prognosis with 10 datasets exhibits that 8 (80 %) are consistent with their histopathological results obtained from the postoperative biopsy.

Fig. 2 Comparison on segmentation results using coupled directional level-set (CDLS) and GrabCut

Fig. 1 The differentiation and staging process: (a) bladder wall segmentation; (b) extracting the tumor from wall tissues; (c) relative invasion depth and the determination of staging

123

Conclusion The proposed scheme in this study not only improves the accuracy and the efficiency of bladder wall segmentation, but also provides a promising way to differentiate bladder carcinomas from wall tissues for further evaluation of invasion depth and cancer staging. Although the preliminary results look promising, only 10 datasets were involved in this study. Further efforts should be devoted to enroll more datasets for intensive assessment of the proposed scheme. Acknoledgement This work was partially supported by National Natural Science Foundation of China under Grant No. 81230035, the Shaanxi Provincial Foundation for Social Development and Key Technology under grant No. 2015SF177, and Guangdong Innovative Research Team Program of China under grant No. 2011S013. References [1] Torre L, et al., Global cancer statistics. CA: A Cancer Journal for Clinicians, pp.1–22, 2015.

Int J CARS [2] [3] [4]

Rother C, et al., Grabcut: Interactive foreground extraction using iterated graph cuts, ACM TOG, 23(3): pp.309–314, 2004. Boykov Y, et al., Fast Approximate Energy Minimization via Graph Cuts, In ICCV, 1, pp.377–384, 1999. Xiao D, et al., 3D detection and extraction of bladder tumors via MR virtual cystoscopy, Int. J. CARS, pp. 1–8, 2015.

Improving soft tissue segmentation in CT volumes using a sigmoid-based active shape model Mina Esfandiarkhani1, Amir Hossein Foruzan1, Yen-Wei Chen2 1 Department of Biomedical Engineering, Engineering Faculty, Shahed University, Tehran, Iran 2 Intelligent Image Processing Lab, College of Information Science and Engineering, Ritsumeikan University, Shiga, Japan Keywords Active shape model  Sigmoid edge model  Liver segmentation  Medical image processing. Purpose The success of Active Shape Models is largely dependent on accurate assignment of point correspondences. Improper labeling of shape points results in unsmooth shape and inaccurate appearance models. This point is vital especially in segmentation of soft tissues with large shape variations. We propose a generalized edge model and employ it in the search algorithm to improve segmentation of liver in the presence of noise and Partial Volume Effect (PVE). Also, we tackle constraints imposed by Gaussian intensity profiles in the conventional ASM [1]. Sigmoid function is used to model the gray-level profiles and classify shape points into genuine and dubious groups. Methods We employ the ‘‘Narrow Band Thresholding’’ technique [2] to initially segment the liver. Initial boundary of a slice is represented by a set of points. In order to speed up our algorithm, the number of points is reduced based on their distance and curvature. We reduce number of points in low-curvature regions of the contour. Next, intensity profiles are sampled along the normal vector in each point. An intensity profile is modeled by a sigmoid function and the parameters of the model are used to estimate the location of the bordering point. Based on the quality of model fitness which is measured quantitatively, we assign a confidence weight to a point representing how well a sigmoid function fits the current profile. If the quality of fitness is below a threshold, the point will be labeled as a dubious point. It means that the boundary point is not located on a strong edge (due to partial volume effect or noise). Thereby, boundary points are clustered into true and false boundary points. Using the position of true boundary points (genuine points), we estimate the position of false boundary points through fitting a cubic Smooth Spline [3]. To improve the results, we employ a statistical shape model and constrain the shape by the model. Results We applied our method to two different CT datasets. The first dataset belonged to Osaka University Hospital, Japan. It contained 30 abnormal abdominal CT images of the second phase with a resolution and size of 512 9 512 9 159 and 0.63 9 0.63 9 1.25 mm3, respectively. The second set included 20 liver CT images of MICCAI 2007 Grand Challenge (Fig. 1).

Fig. 1 Typical visualizations of our results. Segmented livers by (a) conventional ASM, (b) proposed method and (c) gold standard. (d) Boundaries of conventional ASM (green), proposed method (blue) and gold standard (red) in a typical slice. (e) Surface rendering of two typical livers. Left column: Osaka dataset, no. 2. Right column: MICCAI dataset, no. 4. Rows from top to bottom: The proposed, manual and Active Contour Model results We evaluated the proposed method using Dice, Jaccard and MICCAI 2007 Grand Challenge metrics [4]. Quantitative evaluations of our method for both groups are given in Table 1. Table 1 Quantitative evaluations of our results Dataset

Osaka

Signed Avg. Sym. Avg. Sym. Max Vol. Rel. Vol. Surf. Dist. RMS Surf. Surf. Dist. Overlap Diff. [%] [mm] Dist. [mm] [mm] Err. [%] -0.75

1.85

3.01

21.56

12.1

MICCAI -0.09 training dataset

1.82

3.61

30.12

10.29

The average Dice (Jaccard) metrics are 0.93 (0.87) and 0.94 (0.89) for the 1st and 2nd dataset, respectively. The indices are 0.88(0.76) and 0.85 (0.73) for Active Contour Model (ACM), conventional ASM, respectively. The results are improved at least 0.05 and 0.08 with respect to conventional ASM and ACM methods. Conclusion We proposed a generalized edge model which alleviates problems concerned with profile modeling and inaccurate point correspondences in the ASM algorithm. It is robust to noise and PVE and does not leakage to nearby organs. In future, we decide to dynamically change the threshold of genuine and dubious landmarks selection. References [1] Cootes, T.F., Taylor, C.J., D.H. Cooper and Graham, J., ‘‘Active shape models—their training and application,’’ Computer Vision and Image Understanding, 61(1), pp. 38–59, Jan. 1995. [2] Foruzan, A. H., Yen-Wei, C. H. E. N., Zoroofi, R. A., Furukawa, A., Masatoshi, H. O. R. I. and TOMIYAMA, N. (2013).

123

Int J CARS

[3] [4]

Segmentation of liver in low-contrast images using K-means clustering and geodesic active contour algorithms. IEICE TRANSACTIONS on Information and Systems, 96(4), 798–807. Ha¨rdle, W. and Linton, O. (1994). Applied nonparametric methods. Handbook of econometrics, 4, pp. 2295–2339. Van Ginneken, B., Heimann, T. and Styner, M. (2007). 3D segmentation in the clinic: A grand challenge. 3D segmentation in the clinic: a grand challenge, pp. 7–15.

Supervised Hessian-based vessels segmentation in narrow-band laryngeal images S. Moccia1,2, E. De Momi2, A. Ghilardi2, A. Lad2, L. S. Mattos1 1 Istituto Italiano di Tecnologia, Department of Advanced Robotics, Genova, Italy 2 Politecnico di Milano, Department of Electronics, Information and Bioengineering, Milano, Italy Keywords Narrow-band laryngoscopy  Laryngeal tumor  Vesselness measure  Supervised vessel segmentation Purpose The stage in which a laryngeal tumor is diagnosed has strongly impact on patients’ mortality or after treatment morbidity. The current diagnosis practice requires sampling suspicious laryngeal tissue, identified through endoscopic examination, for subsequent histopathological analysis. During the endoscopy, clinicians mainly focus on identifying vascular pattern modifications, which are well known to correlate to cancer onset. Recent developments in this field have led to the introduction of narrow-band (NB) endoscopy. NB enhances the visualization of superficial vessels, thus improving the detection of tumors, especially at early stages, when the diagnosis is challenging but crucial for the patient [1]. In this context, a computer-assisted scheme has the potential to greatly improve tissue evaluation during the diagnosis, increasing the efficiency in managing patients and leading to potentially enormous benefits, especially resulting from an improved detection of early tumors, which may pass unnoticed to the human eye. Methods The proposed method consists in three main steps: (i) Pre-processing, (ii) Vessel enhancement and (iii) Vessel segmentation. The pre-processing step concerns noise suppression, mainly associated to CMOS and CCD image sensors, and edge enhancement. In this work, a non-linear anisotropic rotation invariant diffusion scheme [2] was used. The key idea is to smooth non-informative homogeneous areas with an isotropic Gaussian-like kernel and enhance meaningful edges by an anisotropic kernel, elongated in the direction parallel to the edge itself. The two eigenvectors of the structure tensor J were employed to estimate the edges direction, while the amount of diffusion was defined by a combination of J eigenvalues, defined to preserve both plate-like and tubular-like structure. A second issue is related to the presence of specular reflections (SR), due to the strong illumination of the endoscope and the wet and smooth surface of laryngeal tissue. SR have to be identified and masked, since they represent a source of errors for the vessel segmentation algorithm. SR segmentation was performed with a thresholding scheme on both the saturation (low for SR) and brightness (high for SR). The two thresholds were set as the best compromise between sensitivity and specificity after a receiver operative characteristic (ROC) curve analysis. The vessel enhancement was performed with the image Hessian eigenvalues (k1, k2 with |k1| B |k2|) analysis. H was computed

123

according to the scale-space theory [3], namely convolving the image with the second partial derivatives of a Gaussian kernel with scale r. Since laryngeal vessels can assume both healthy tubular-like (|k1|  |k2|) and pathological blob-like ((|k1| & |k2|) structure, only the amplitude of k2 was considered. Moreover, only negative k2 were retained because vessels in NB images are darker than the background. Different r values were considered to take in account for vessels with different thickness, resulting in a multi-scale framework analysis. The multi-scale vesselness measures computed in the previous step were used as input features to a linear support vector machine (SVM) classifier to obtain vasculature segmentation. The input features were normalized to obtain variables with zero-mean and unit standard deviation, according to the approach in [4]. Two classes have been considered during the classification process: vessel and background class. Through this classification of pixels, the final vessels segmentation was achieved. Results A dataset of twenty pathological NB laryngeal frames was used in this research, with an image size of 478x311 pixels. Ten images were used to tune the algorithm parameters and train the SVM, while the remaining ten were used to test the proposed algorithm performance with respect to gold-standard manual segmentations performed by an expert. The adopted metrics were the area under the ROC curve for the enhancement step, and accuracy, sensitivity and specificity for the segmentation step. The proposed approach performed better (AUROC [ 0.892) when compared with other methods in the literature (AUROC \ 0.867). The introduction of the denoising step further improved the algorithm performance. The segmentation sensitivity, specificity and accuracy were 0.612, 0.933 and 0.895, respectively. Results from the processing of one test image are illustrated in Fig. 1.

Fig. 1 Original laryngeal image (a). Vessel enhancement for r = 1.3 (b) and 1.5 (c). Vessel segmentation result (d) Conclusion In this work, a fully automatic laryngeal vessel segmentation algorithm has been proposed. The anisotropic nature of the implemented denoising algorithm proved to be effective to face the noisy nature of endoscopic images while enhancing meaningful features. This was demonstrated by the higher performance reached in the subsequent vessel segmentation phase. The proposed multi-scale vessel enhancement algorithm outperformed other methods presented in literature in terms of area under the ROC curve, being able to segment both healthy tubular-like vessels and pathological blob-like vessels. In addition, these results demonstrate the benefits of the SVM method used, showing it was able to efficiently model the complexity of endoscopic images in terms of noise, non-uniform illumination and non-constant vesselness measure. Future developments will focus on improving the classification algorithm through an enlarged training and evaluation dataset, which will include a wider range of laryngeal pathologies. References [1] Lukes P, Zabrodsky M, Lukesova E, Chovanec M, Astl J, Betka J A, Plzak J (2014) The role of nbi hdtv magnifying endoscopy in the prehistologic diagnosis of laryngeal papillomatosis and spinocellular cancer. BioMed research international. [2] Mendrik A M, Vonken E, Rutten A, Viergever M A, Van Ginneken B (2009) Noise reduction in computed tomography

Int J CARS

[3] [4]

scans using 3D anisotropic hybrid diffusion with continuous switch. Medical Imaging, IEEE Transactions on 28(10): 1585–1594. Koenderink J J (1984) The structure of images. Biological cybernetics 50(5), 363–370. Soares J V, Leandro J J, Cesar Jr R M, Jelinek H F, Cree M J (2006) Retinal vessel segmentation using the 2D Gabor wavelet and supervised classification. Medical Imaging, IEEE Transactions on 25(9), 1214–1222.

was marked as area of interest. Then, an additional registration step established correspondences between the 400 N and the two other images. Each bone was registered separately (rigididly, using the normalized gradient fields distance measure within a dilated region capturing the boundary of the respective bone only), so that the loadinduced joint movement could be appropriately captured.

Image processing pipeline for MRI-based in-vivo cartilage assessment under load H. Meine1, K. Izadpanah2, T. Lange3 1 University of Bremen, Medical Image Computing, Bremen, Germany 1 Fraunhofer MEVIS, Bremen, Germany 2 Univ. Medical Center Freiburg, Orthopedic and Trauma Surgery, Freiburg, Germany 3 Univ. Medical Center Freiburg, Radiology, Medical Physics, Freiburg, Germany Keywords Patellofemoral joint  Mechanical loading  Cartilage thickness  Contact mechanism Purpose Cartilage loading conditions are believed to play an important role in initiation and progression of osteoarthritis (OA), a widespread disease. However, there exists very little knowledge about the response of the patellofemoral cartilage to acute loading [1, 2], even in a healthy population. We introduce the first complete set-up for comparative in vivo patellofemoral cartilage morphology assessment under controlled loading conditions. Methods We developed a novel technique for in vivo assessment of the patellofemoral cartilage under in situ mechanical loading. This system is based on MR imaging, an MR-compatible mechanical loading device, prospective motion correction, and a dedicated image processing pipeline. In the following, our focus is on the image analysis. We used a slightly improved version of our previously described imaging setup [3]. Every subject got three MRI scans (T1-weighted, fat-suppressed, FOV 150x122x44 mm, resolution 0.4x0.4x0.5 mm): first without, and then with 200 N and 400 N of load applied pneumatically on the knee, respectively. The MR sequences included a tracking-based prospective motion correction. Within every MR volume, the Patella, the Femur, and the associated cartilage were segmented. Our pipeline started with a semiautomatic bone segmentation, using a marker-based watershed segmentation and a subsequent graph cut-based mesh-optimization step that snapped the surface to the maximum gradient in normal direction [4]. Cartilage surface meshes were then computed based on contours drawn manually on relevant slices, re-using the bone surface from the previous step as the cartilage-bone-interface. For every point on the cartilage surface, the local cartilage thickness (LCT) was defined as the distance to the bone surface [5], a color-coding of which can be seen in Fig. 1a. During evaluation, changes of the LCT were investigated, averaged over the area of contact. Similarly, the patellofemoral contact distance (PFCD) was defined as the distance to the opposite cartilage. Fig. 1b visualizes the PFCD with a special colormap that highlights the area where PFCD \ 1 mm, the second parameter included in the evaluation. In order assess the effect of mechanical loading on these two parameters, measurements were required in exactly the same locations: First, the contact area in the 400 N images

Fig. 1 Color-coded LCT (left), PFCD (center), and changes of contact area under load (right) Results We successfully used our setup to assess load-induced changes to articular cartilage in the patellofemoral joint of 14 healthy subjects under controlled loading conditions. We observed a statistically significant (p \ 0.05 in the Wilcoxon signed-rank test) reduction of the mean patellar cartilage thickness in the area of contact from an average of 3.4 mm over all subjects by 0.11 mm under medium load (200 N) and by 0.23 mm under high load (400 N), and for the femoral cartilage (average 2.73 mm) by 0.13 mm and 0.27 mm. At the same time, the contact area increased significantly under load, in this population from a mean of 6.1 cm2 by an average of 0.53 cm2. No significant difference in contact area was measured between the two loaded situations (200 vs. 400 N). Conclusion The novelty in the presented system for in vivo assessment of cartilage properties lies in the comparative measurements under defined acute loading conditions. This required both highly precise thickness measurements that allow to show effects below voxel resolution, as well as an accurate registration of the images in order to account for bone motion. For the first time, we were able to quantify in vivo cartilage compression in the patellofemoral joint under mechanical loading, by measuring changes in cartilage thickness and volume. At the same time, we could measure an increasing area of contact between the opposing cartilage surfaces. Our preliminary study is ongoing, and we will soon publish the clinically relevant results from a larger population of healthy subjects. Given the insights we gathered from this, a future study shall be conducted with patients showing symptoms of OA or abnormal patella movement, for instance (Fig. 2).

Fig. 2 Observed changes in the measured parameters under low / high load (200 N/400 N)

123

Int J CARS References [1] Nishii T, Kuroda K, Matsuoka Y, Sahara T, Yoshikawa H (2008) Change in knee cartilage t2 in response to mechanical loading. Magnetic Resonance Imaging 28(1):175–180. [2] Souza RB, Stehling C et al. (2010) The effects of acute loading on t1rho and t2 relaxation times of tibiofemoral articular cartilage. Osteoarthritis and Cartilage 18(12):1557–1563. [3] Lange T, Maclaren J et al. (2014) Knee cartilage MRI with in situ mechanical loading using prospective motion correction. Magnetic Resonance in Medicine 71(2):516–523. [4] Ba¨hnisch C, Stelldinger P, Ko¨the U (2009) Fast and accurate 3D edge detection for surface reconstruction. In: Proc. 31st DAGM, Springer, pp 111–120. [5] Lo¨sch A, Eckstein F et al. (1997) A non-invasive technique for 3d assessment of articular cartilage thickness based on MRI part 1. Magnetic Resonance Imaging 15(7):795–804.

Constrained piecewise rigid 2D-3D registration for patient-specific analysis of rib cage motion using X-ray video Y. Hiasa1, Y. Otake1, R. Tanaka2, F. Yokota1, S. Sanada2, Y. Sato1 1 Nara Institute of Science and Technology, Graduate School of Information Science, Ikoma, Japan 2 Kanazawa University, Department of Quantum Medical Technology, Kanazawa, Japan Keywords 2D–3D registration  X-ray video  Rib motion measurement  Respiratory system analysis Purpose The respiratory function has been commonly evaluated by a spirometer in clinical routine. Although the spirometer is non-invasive and provides useful information about overall respiratory function, diseases caused by local lung dysfunction are not easy to diagnose. Tanaka et al. [1] demonstrated that two-dimensional image analysis using a low-dose X-ray video examination in scoliosis patients can reveal the cause-and-effect relationship between degradation of respiratory function and motion of the ribs. The threedimensional rib motion has been studied using a multi-time-phase CT, but increasing time resolution is difficult as a routine examination due to the amount of patient dose. The purpose of this study was to provide an alternative rib motion diagnosis tool that achieves higher time resolution and three-dimensional analysis while keeping the radiation dose at the conventional level. For this purpose, we developed a robust 2D-3D registration algorithm between X-ray video and a one-time-phase CT using constraints based on the anatomical knowledge of the articulation at the costovertebral joint. Methods The framework underlying the proposed 2D-3D registration method follows that of Otake et al. [2] in which an optimization algorithm seeks rigid transformation parameters that maximize the similarity between the radiograph and digitally reconstructed radiograph (DRR). We extended the framework in this work to estimate relative positions of multiple rigid objects connected by joints such as the rib cage. The motion of the rib cage is measured by applying the registration on an X-ray video frame-by-frame (see Fig. 1). The anatomical knowledge between the rib and its neighboring vertebra is used as a constraint to reduce the degree of freedom (DoF) in the optimization. De Troyer et al. proposed a model [3] that approximated the movement of each rib with a uniaxial rotation about an axis connecting the costotransverse and costovertebral joints. The model was validated by Ito et al. by analyzing CTs at the inhale and exhale phases of one subject. We employed this model as the constraint in the 2D-3D registration. First, we segment the outer surface of each rib bone and vertebra in CT and

123

manually identified the costotransverse and costovertebral joints on each rib to define the rotation axis. Thus, in the optimization, the movement of a rib is expressed by one parameter, which is the rotation angle around this axis (i.e., N parameters for N ribs). Together with the six DoF for translation and rotation of the spine, the number of optimization parameter in the proposed method amounts to (6 + N).

Fig. 1 Flowchart for the proposed constrained piecewise rigid 2D-3D registration method Results We conducted two experiments to evaluate the proposed method: simulation experiments and experiments using the real X-ray video. In the simulation experiments, we used DRRs of the CT of two timephase, maximum inhale and maximum exhale, included in the EMPIRE10 data set [4] as the target image. The feasibility of the proposed model was evaluated by comparing the registration error for the two scenarios: (1) the ‘‘true rotation axes’’ were used in the optimization and (2) the axes defined by the anatomical landmarks were used. Here, the ‘‘true rotation axes’’ were calculated by registering each rib in 3D at the two time-phases. The (6 + N) registration parameters were perturbed by a uniform random distribution with the range for translation and rotation of the spine: ± 50 [mm], ± 10 [deg] and rotation angle of the rib: ± 10 [deg]. The results of an evaluation experiment of the 2nd right rib conducted using the error metrics commonly used in [5] suggest more than 95 % in 126 trials resulted in less than 3 [mm] mPD and less than 5 [mm] mTRE. In the real image experiments, we used X-ray videos acquired at the standing position and CTs acquired at the supine position at inspiratory phase of ten cases. The X-ray video included 30 frames acquired over one breathing cycle in ten seconds. An example analysis of the local movement of the rib cage is shown in Fig. 2. That shows an example three-dimensional visualization of the trajectory of the tip of each rib for one breathing cycle.

Fig. 2 Example three-dimensional visualization of trajectory of the tip of each rib for one breathing cycle. Colormap of the trajectory indicates temporal transition from maximum inhale (blue) to maximum exhale (red). The bone at maximum inhale is shown in cream and the maximum exhale in wine red

Int J CARS Conclusion In this study, we proposed a system to analyze the respiratory function by measuring local movement of patient’s rib cage three-dimensionally from an X-ray video with a high time resolution. Evaluation experiments were conducted with both generated images and real images. The simulation study with the two scenario suggested that the rotation axis defined by the manually identified landmarks provided the sufficient registration accuracy comparable to the scenarios where the true rotation center is known. The experiment with real images demonstrated feasibility of the proposed method with the X-ray video acquired by an actual clinical protocol. We have investigated the visualization method to realize a quantitative evaluation of patientspecific respiratory function. Development of a more accurate anatomical model considering deformation of spine curvature between the supine position (at the CT acquisition) and the standing position (at the X-ray video acquisition) is underway. Additional constraints such as function of intercostal muscles and continuity over time will be also included in our future work. References [1] Tanaka R, et al. (2015) Quantitative analysis of rib kinematics based on dynamic chest bone images: preliminary results. J. Med. Imag. 2(2):1–8. [2] Otake Y, et al. (2012) Intraoperative image-based multiview 2D/ 3D registration for image-guided orthopaedic surgery: incorporation of fiducial-based C-arm tracking and GPU-acceleration. IEEE Trans. Med. Imag. 31(4):948–962. [3] Troyer De, et al. (2005) Respiratory action of the intercostal muscles. Phy. Rev. 85(2):717–756. [4] Murphy K, et al. (2011) Evaluation of registration methods on thoracic CT: the EMPIRE10 challenge. IEEE Trans. Med. Imag. 30(11):1901–1920. [5] Kraats D, et al. (2005) Standardized evaluation methodology for 2-D-3-D registration. IEEE Trans. Med. Imag. 24(9):1177–1189.

Brain atrophy measurement employing non-rigid registration using free form deformation based on de-boor control points M. Ghodrati1, E. Fatemizadeh2 1 Islamic Azad University Science and Research Branch, Biomedical Engineering, Tehran, Iran 2 Sharif University of Technology, Department of Electrical Engineering, Tehran, Iran Keywords Brain atrophy measurement  Image registration  FFD  De Boor Points Purpose Brain atrophy is one of the common markers in many neuro-degenerative processes such as Alzheimer’s disease and Multiple Sclerosis (MS). Several methods have been proposed for brain atrophy measurement, among which Jacobian Integration (JI) has proven to be one of the most precise [1,2]. This method uses the non-rigid Free Form Deformation (FFD) registration based on B-Spline basis functions [3] in which the control points are the parameters. The determinant of the Jacobian matrix at each voxel is used to calculate the whole atrophy [4]. Measuring the exact amount of small atrophies occurring within the brain, requires very precise methods. This paper aims to optimize the JI algorithm. Because this method is based on image registration, some changes in the image registration algorithm are made, in particular replacing De-Boor points with B-spline control points. Methods The connection between De-Boor points and B-Spline control points are defined in Fig. 1: where the first column fP0 ; P1 ; . . .; Pn g

represents B-Spline control points and the other columns are De-Boor points at different depths.

Fig. 1 Relation between B-Spline control points and D-Spline points This correspondence leads to a new curve using 2 sets of De-Boor points   P0 ; P11 ; P22 ; . . .; Pnn to represent the first set of new control points and  n n1  Pn ; Pn ; . . .; P1n ; Pn to represent the second set of new control points. These two groups of points are then used to replace B-Spline control points, resulting in a new curve as follows, C ðt Þ ¼

n   1X Bi;k ðtÞPii ðtÞ þ Bi;k ðtÞPni n ðtÞ 2 i¼0

in which Bi;k ðtÞ are B-Spline basis functions and Pii ðtÞ and Pni n ðtÞ are De-Boor control points. This equation results new basis functions (DSpline basis functions) which have properties identical to B-Spline basis functions. Results Some interpolation examples using the two spline methods were employed to compare their accuracy. The D-Spline method, in comparison to the B-spline method, gives better results in interpolating curves and surfaces. Using the D-Spline method, the error in interpolating curves decreased by more than two-fold (from 2.98 to 1.26) and the error in interpolating surfaces by almost four-fold (129.04 to 35.42). The evaluation of the two spline methods in atrophy measurement was done in two steps. First, 7 MRI images from BrainWeb were compared to similar images uniformly reduced by 3 % Fig. 2., i.e. to 0.97 of original size in each dimension to approximate atrophy typical of an AD patient over one year [5].

Fig. 2 Sagittal view of Left: BrainWeb image data and Right: Result of volume reduction The two Spline methods were applied to these images. The results suggested that the D-Spline method was able to recognize 2.25 % and the B-Spline method about 2 % of the imposed volume reduction

123

Int J CARS Table 1. As a second step, atrophy was measured in 20 images of patients diagnosed with Alzheimer’s disease obtained from the ADNI website. Volume reduction were calculated using both spline methods. The B-Spline method showed 2.06 % and the D-Spline method 2.21 % volume reduction. Table 1 Means (SD) of AD subjects over 1 year and brain volume reduction in BrainWeb data Recognized Volume Reduction of 3 %

AD (%/year) atrophy

JI (B-Spline)

2.00

2.06

JI (D-Spline)

2.25

2.21

Conclusion Results show that the D-Spline method performs better in both curve and surface interpolation. In comparing the two spline methods in brain atrophy measurement, the D-Spline method was significantly more accurate. References [1] Cash M, Frost C (2015) Assessing atrophy measurement techniques in dementia: Results from the MIRAD atrophy challenge. NeuroImage 123, 149–164. [2] Durand D, et al. (2012) Reliability of Longitudinal Brain Volume Loss Measurements between 2 Sites in Patients with Multiple Sclerosis: Comparison of 7 Quantification Techniques. AJNR Am Neuroradiol. Nov;33(10):1918–24. [3] Rueckert, D., Sonoda, L.I., 1999. Nonrigid registration using free-form deformations: appli- cation to breast MR images. IEEE Trans. Med. Imag. 18, 712–721. [4] Boyes, R., Rueckert, D., (2006) Cerebral atrophy measurements using Jacobian integration: comparison with the Boundary Shift Integral. NeuroImage 32, 159–169. [5] Sharma, S., Noblet, V., (2010) Evaluation of brain atrophy estimation algorithms using simulated ground-truth data. Medical Image Analysis 14, 373–389.

anatomical deformation, image content mismatch (surgical devices present in the intraoperative image but not the preoperative), and large capture-range while extending the approach to increasingly common scenarios in which MRI (not CT) is used for preoperative visualization and planning. Direct extension of the previous method is confounded by large mismatch in image-intensity and tissue correspondence between MRI and radiographs, but the work reported below overcomes such challenges using vertebrae segmentation. This work presents the methodology and first evaluation of feasibility and performance. Methods Clinical images were collected in an IRB-approved study involving 5 patients undergoing thorocolumbar spine surgery following a preoperative MRI. There were no changes to standard-of-care imaging protocols for either the preoperative 3D images (sagittal T2-weighted MRI) or intraoperative 2D images (lateral mobile radiographs). As with conventional LevelCheck, the centroids of vertebral labels are annotated manually in the preoperative MRI using standard designations (C1-S1) to be projected onto the radiographs. Providing these vertebral centroids as input, we developed an automatic segmentation algorithm to extract vertebrae boundaries from MRI to be subsequently used in registration. Image segmentation was formulated as a spatially continuous min-cut problem and solved globally and exactly as described in [3] to obtain the segmentation output. 3D-2D registration involves iteratively alignment of MR projections (or segmented structures therein) with the radiograph by optimizing image-similarity. Due to strong differences in MRI and radiographic image-intensities, obtaining a radiograph-like image from MRI projections is not straightforward. Using the segmentation output, however, we investigated four approaches to generate MR projections: the p1 method projects the original MR intensities (T2-weighted signal values) within each segmented vertebrae; the p2 approach first dilated the segmentation to include a region approximating the bony cortex and then projected as in p1; the p3 method projected the p1 segmentation as a binary region (disregarding image features internal to the segmentation); and the p4 method projected the vertebral body as in p3, dilated to include the bone cortex. In each case, 3D-2D rigid registration was performed as in Fig. 1 using gradient orientation (GO) as a similarity metric and the covariance-matrix-adaptation-evolutionarystrategy (CMA-ES) optimizer as described in [4]. By perturbing this manual initialization within a range ± 100 mm, 10 repeated registrations were performed for each image-pair.

‘‘LevelCheck’’ localization of spinal vertebrae in intraoperative radiographs from preoperative MRI T. De Silva1, A. Uneri2, M. Ketcha3, S. Reaungamornrat2, J. Goerres3, S. Vogt4, G. Kleinszig4, J.- P. Wolinsky5, J. Siewerdsen1 1 Johns Hopkins University, Department of Biomedical Engineering, Baltimore, United States 2 Johns Hopkins University, Department of Computer Science, Baltimore, United States 3 Johns Hopkins University, Biomedical Engineering, Baltimore, United States 4 Siemens Healthcare XP Division, Erlangen, Germany 5 The Johns Hopkins Hospital, Department of Neurosurgery, Baltimore, United States Keywords LevelCheck  3D-2D registration  Image-guided surgery  Target localization Purpose Previous work introduced the LevelCheck [1–2] algorithm for localizing spinal vertebrae in intraoperative radiographs from preoperative CT based on robust 3D-2D registration. The work described below extends the method to operation based on preoperative MRI. The method maintains desirable properties of robustness against

123

Preoperave 3D image

Vertebrae

Intraoperave e 2D image

Labelled radiograph

T12

L1

CMA-ES opmizer

MR projecon

L2 L3

L4 L5

GO metric S1

Fig. 1 3D-2D registration workflow of MR LevelCheck algorithm using a gradient orientation similarity metric and CMA-ES optimizer

Int J CARS The accuracy and robustness of the segmentation method was evaluated by comparing to manually annotated segmentations of vertebral bodies and computing the Dice-coefficient, mean-absolute-distance (MAD), and maximum-absolute-distance (MAXD). Registration accuracy was evaluated with respect to the task of vertebrae localization in the radiograph: ground truth vertebral locations defined in the radiographs were compared with the 3D-2D registration output to calculate projection distance error (PDE). Failure was defined as PDE [ 30 mm (a distance for which the registered label may lie outside the vertebral body). Results Nominal parameters for segmentation regularization were evaluated in a sensitivity analysis. With the nominal settings, the segmentation accuracy across 5 patients was: Dice coefficient = 89.2 ± 2.3 (mean ± stdev); MAD = 1.5 ± 0.3 mm; and MAXD = 5.6 ± 0.7 mm. The Dice and MAD results are comparable to other MR spine segmentation methods reported in literature [5]. The larger MAXD is attributed to nominal regularization parameter that emphasized distinct identification of the vertebrae boundary with some occasional protrusions at the vertebrae pedicle. Violin plots for the distributions in PDE for 5 patient registrations (each perturbed 10 times in initialization) are shown in Fig. 2. Performance is shown for the four projection methods: for the p1 method, PDE = 27.1 ± 14.6 mm (median ± iqr) with a 36 % failure rate; the p2 method improved to PDE = 5.9 ± 46.7 mm but suffered in robustness and was also subject to 40 % failure rate; the p3 method (projection of the binary segmentation) improved PDE to 4.9 ± 3.2 mm with only 10 % failure rate; and the p4 method (projection of the binary cortex) yielded the best performance, with PDE = 4.8 ± 0.8 mm and all registrations converging at the desired solution. Run-time was 23.3 ± 1.7 s (mean ± iqr).

LevelCheck as initial seed, thus adding no additional workflow to the process. Ongoing work includes validating the method in a larger clinical dataset and analyzing the sensitivity of registration to the quality of automatic segmentation. References [1] Otake Y. et al., ‘‘3D-2D registration in mobile radiographs: algorithm development and preliminary clinical evaluation.,’’ Phys. Med. Biol., vol. 60, no. 5, pp. 2075–90, 2015. [2] Lo S-FL. et al., ‘‘Automatic localization of target vertebrae in spine surgery: clinical evaluation of the LevelCheck registration algorithm.,’’ Spine (Phila. Pa. 1976)., vol. 40, no. 8, pp. E476–83, 2015. [3] Yuan J. et al., ‘‘A continuous max-flow approach to Potts model,’’ Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 6316 LNCS, no. PART 6, pp. 379–392, 2010. [4] De Silva T. et al., ‘‘3D-2D Image Registration for Target Localization in Spine Surgery: Investigation of Similarity Metrics Providing Robustness to Content Mismatch,’’ Phys. Med. Biol., 61 (8), 3009–3025, 2016. [5] Neubert A. et al., ‘‘Automated detection, 3D segmentation and analysis of high resolution spine MR images using statistical shape models.,’’ Phys. Med. Biol., vol. 57, no. 24, pp. 8357–76, 2012.

2D to 3D registration of manually segmented MRI prostate data B. Maris1, P. Fiorini1 1 University of Verona, Computer Science, Verona, Italy Keywords MRI images  Prostate treatment follow-up  Prostate manual segmentation  Contour registration

Fig. 2 Registration performance. Violin plots showing PDE distributions for four projection methods. ‘‘Failure’’ line (red) at PDE = 30 mm demarks the threshold for which the registered label is likely outside the true vertebra Conclusion This work presented an important extension of the previously reported LevelCheck algorithm for use with preoperative MRI (c.f., CT). The method leverages the same underlying framework for robust, highspeed 3D-2D registration and overcomes dissimilarities in image intensities, gradients, and tissue correspondences through simple segmentation of vertebrae in MRI. The segmentation was automatic and used the same 3D vertebral label as normally defined in

Purpose Image guidance and navigation systems for prostate biopsy or minimally invasive prostate treatment rely on the precise localization of the instrument position (needle or probe) and of the lesion inside the prostate. One of the main limitation of existing systems is the fact that they are based on external tracking that take into account only the movement of the imaging device or of the interventional tool, while the target can change in position and/or shape. Since most of the methods for prostate registration are based on the previous segmentation of the data, we propose here a method that register complete or partial manually segmented planar contours with a complete segmentation of the organ. The algorithm yields the position of the complete/partial segmented slice inside the whole dataset and gives the planar transformation (rotation and translation) between the current slice and the reference dataset. The method works when little or no deformation is present. In this work we are interested on aligning two MRI datasets of the same patient taken in different moments in time: before and after prostate treatment in order to monitor the therapy. Both datasets are acquired with the patient placed in the same position. Methods The registration method is based on the manual segmentation of an image slide. The segmentation is done by choosing points on the boundary of the prostate. The selection of the points is arbitrary, therefore an ordering is required. When the contour is complete we have implemented a method for the point ordering as in [1], while when the contour is incomplete we simply project on the circle that fit best the partial segmentation and order the points according to the correspondent circle location. The second method is simpler and works in most of the cases when the contour is convex or almost convex even with complete segmentation.

123

Int J CARS When the points are ordered, we can parameterize the initial segmentation using splines as an open or closed curve which becomes an affine map between the external segmentation in plane and a linear segment. By doing the same operation on another planar path of points that is part of the 3D segmentation of the prostate we shall have another affine map of the curve into a linear segment. The next step is to find points along each of the curves at equal distance, that is if A are the points on the first curve and B are the points on the second curve, we must have that the distances between each two consecutive points of A or B must be equal (condition 1) and the distances between two consecutive points in A and two consecutive points in B must be equal (condition 2). (condition 1) may be obtained by computing the ration between the length of the curves and then setting an appropriate value for the distance between two consecutive points. Once we have defined this distance we formulate the problem of the computation of the point sets A and B in terms of differential equations that describe the path along the curve. Then the interpolation can be done using an ODE solver and spline parameterization. Now that we have a regular sampling on each of the curves, we define the first curve as the reference and the second curve as target (see Fig. 1). It is obvious that the target curve has less points than the reference. Starting from each point of the reference set we build a circular buffer that has the cardinality of the target set (upper row in the Fig. 1). The two sets with the same cardinality may be registered by using a least square closed form solution as in [2].

Conclusion In this work we have introduced a method to find the position of a slice inside a larger dataset. The registration is based on the manual segmentation in plane of the surface of the prostate. The algorithm works by simply selecting points on a slice, a process by which radiologists are accustomed on every day practice. This is a preliminary work that will be extended to suit more complex scenarios where the use of other systems is envisioned (e.g. images obtained with an ultrasound probe and segmented manually, MRI intra-operative images and other tracked tools that may allow, after the 2D-3D registration, the positioning of the pre-operative dataset with respect to the patient in the operating room). The ultrasound-MRI registration will be our main interest since are two complementary methods for the prostate biopsy and for minimally invasive treatment. Our algorithm will then identify the position of the ultrasound slice inside the MRI dataset by simply using points on the contour of the prostate identified by the physician. The position of the ultrasound image is then calibrated with the biopsy needle or the therapy probe. We investigate also the extension of the method to use 3D surfaces instead of 2D contours. References [1] Ohrhallinger S, Mudur SP ‘‘Interpolating an unorganized 2D point cloud with a single closed shape,’’ Comput. Des., vol. 43, Dec. 2011. [2] Arun KS, Huang TS, Blostein SD ‘‘Least-Squares Fitting of Two 3-D Point Sets,’’ IEEE Trans. Pattern Anal. Mach. Intell., pp. 698–700, 1987.

Deformable registration of ultrasound and MRI using a new self-similarity based neighborhood descriptor D. S. Jiang1,2, M. N. Wang1,2, Z. J. Song1,2 1 Fudan University, Digital Medical Research Center of School of Basic Medical Sciences, Shanghai, China 2 Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai, China Keywords Deformable registration  Self-similarity  Ultrasound and MRI registration  Local image descriptor

Fig. 1 Workflow for prostate segmentation Results We have built an intuitive Matlab interface where a physician can load two different datasets of the prostate (before and after the therapy) and can easily perform the segmentation and check the registration result. The first dataset is completely segmented in 3D and will be the reference dataset, while the second will represent the target. Since we use the same z-axis for both datasets we have compared the target segmentation with the planar contour of each intersection of a plane perpendicular to the z-axis to the 3D dataset (see Fig. 1). The target is compared with each section of the reference and the best value of z will represent the position of the target slice inside the reference slice, while in the same time we obtain the inplane rotation and translation between the two contours. In our tests we have obtained an error of less than 2 mm by sampling the z-axis at 1 mm.

123

Purpose Computer aided intervention often depends on multi-modal deformable registration to make full use of different modality images to provide complementary information. However, multi-modal deformable registration remains a challenging task and is an active research area in medical image analysis, especially in the deformable registration between ultrasound (US) and magnetic resonance imaging (MRI). Methods Structural image representations based multimodal image registration has proved to achieve good results compared with mutual information based methods in recent years, such as modality independent neighborhood descriptor (MIND) [1] and self-similarity context (SSC) [2]. Compared with SSC, MIND suffers the noise sensitivity of the central patch, but it estimates the feature of the central voxel directly. Even though SSC overcomes the problem of central patch noise with calculations of all pair-wise distances of patches within the six neighborhoods, it does not directly consider the voxel feature of interest, which makes it a robust descriptor but not sensitive to the local feature. So both descriptors are not perfect. This paper proposed a more robust and sensitive structural image descriptor based on MIND and SSC, which integrates MIND and SSC with a distance weight (MINDWSSC) and transforms the complex multi-modal registration problem to mono-modal registration. The Euclidean distance

Int J CARS between patches was used as the weight, which is 1 for MIND and is pffiffiffi 2 for SSC. Figure 1 illustrates the concept of MINDWSSC. In this way, an 18D vector descriptor for every voxel is generated. Gauss– Newton optimization with sum of square differences as a metric is used to build the registration framework. To preserve topology and avoid implausible folding, a smooth diffeomoprhic transformation is applied.

Fig. 1 Concept of the three descriptors with six neighborhoods. The central patch around the voxel of interest is shown in red, all patches within its immediate six neighborhoods are shown in pink. All patch distances are shown with green lines Results The three descriptors were tested on a set of 13 pairs of pre-operative MRI and pre-resection 3D ultra-sound (US) images of the Brain Images of Tumors for Evaluation (BITE) database [3] from the Montreal Neurological Institute. The TRE and mTRE after initial manual alignment and that after deformable registration with different descriptors are shown in Table 1. The average initial mTRE is 4.39 ± 1.4 mm. MINDWSSC achieves the best overall registration accuracy of 2.29 ± 1.2 mm, compared with MIND of 2.40 ± 1.3 mm and SSC of 2.46 ± 1.3 mm, which proves the robustness and accuracy of this descriptor. Furthermore, we conducted an analysis of a paired two-sample t-tests (the p value between MIND and MINDWSSC is 0.07, the p value between SSC and MINDWSSC is 0.0017), the result shows MINDWSSC outperforms MIND and SSC with 90 % confidence interval. Table 1 TRE and mTRE results of MRI/US registration using MIND, SSC, MINDWSSC. Each entry is mean ± std. All numbers in mm Case

Initial

MIND

SSC

MINDWSSC

2.84 ± 1.6

2.28 ± 1.3

2.25 ± 1.4

P1

6.06 ± 1.6

P2

11.90 ± 2.0

2.26 ± 1.3

2.91 ± 1.7

2.54 ± 1.3

P3 P4

3.90 ± 1.1 3.06 ± 1.5

1.75 ± 0.8 2.17 ± 1.3

1.95 ± 1.0 2.08 ± 1.3

1.74 ± 0.9 1.97 ± 1.3

P5

2.26 ± 1.0

2.11 ± 1.0

2.13 ± 0.9

1.96 ± 0.8

P6

3.29 ± 1.6

2.71 ± 1.4

3.38 ± 1.5

2.89 ± 1.5

P7

4.04 ± 2.0

2.46 ± 1.3

2.44 ± 1.4

2.35 ± 1.2

P8

5.13 ± 1.4

2.50 ± 1.9

2.48 ± 1.4

2.49 ± 1.6

P9

2.77 ± 0.7

1.60 ± 0.8

1.93 ± 0.9

1.70 ± 0.8

P10

1.53 ± 0.7

1.71 ± 0.7

2.10 ± 0.9

1.70 ± 0.7

P11

3.37 ± 1.8

2.79 ± 0.9

2.47 ± 0.9

2.32 ± 0.9

P12

4.39 ± 1.4

2.74 ± 1.4

2.60 ± 1.4

2.70 ± 1.6

P13

5.41 ± 1.4

3.57 ± 1.8

3.20 ± 1.7

3.10 ± 1.7

Mean

4.39 ± 1.4

2.40 ± 1.3

2.46 ± 1.3

2.29 ± 1.2

Acknowledgement This study was supported by NSFC Projects 81471758, 81271670 and MOST Project 2015BAK31B01.

Conclusion A new structural image descriptor based on image self-similarity is presented to deal with the challenging deformable multi-modal image registration of pre-operative MRI and intra-operative US for neurosurgery. Compared with MIND and SSC, MINDWSSC is more robust to noise and more sensitive to local structures with little cost of computation time. Local image representation opens a new door to address the multi-modal deformable registration problem, which is quite different with MI and its variants. However, to utilize the complex high-dimensional vector descriptor to assist deformable registration without increasing computation complex is still worth further study. In the future, a GPU implementation of MINDWSSC and new optimization and deformation modal integrated with MINDWSSC will be investigated to decrease the computation time and improve accuracy. References [1] Heinrich MP, Jenkinson M, Bhushan M, Matin T, Gleeson FV, Brady SM, Schnabel JA (2012) MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration. MED IM-AGE ANAL 16: 1423–1435. [2] Heinrich MP, Jenkinson M, Papie_z BW, Brady M, Schnabel JA (2013) Towards realtime multimodal fusion for image-guided interventions using self-similarities Medical Image Computing and Comput-er-Assisted Intervention–MICCAI 2013 Springer, pp 187–194. [3] Mercier L, Del Maestro RF, Petrecca K, Araujo D, Haegelen C, Collins DL (2012) Online database of clinical MR and ultrasound images of brain tumors. MED PHYS 39: 3253–3261.

Multimodal registration using 3D-FAST conditioned mutual information X. Liu1,2, M. Wang1,2, Z. Song1,2 1 Digital Medical Research Center of School of Basic Medical Science, Fudan University, Shanghai, China 2 Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai, China Keywords Non-rigid  Multimodal  Mutual information  FAST Purpose Mutual information (MI) is widely used as a similarity measure for medical image registration. Its accuracy and robustness have been demonstrated for rigid registration of both unimodal and multimodal images. However, when extended to non-rigid registration of multimodal images, MI has many limitations. This is because MI only considers the global intensity correlation while ignoring local and geometric information [1]. In this paper, we bring local structural information into the calculation of MI to make it more suitable for deformable registration of multimodal images. Methods Our proposed method is denoted as FastMI which takes structural similarity as the conditional information of MI. The structural information used in our method is based on Features from Accelerated Segment Test (FAST). FAST is a fast 2D corner detection algorithm. We first extend the idea of FAST into 3D to detect the largest surface patch on a sphere and calculate a score for each voxel according to the area of the detected surface patch associated with it. We explain 3D-FAST as follows: 1. Select a voxel 9 in overlap regions of the two images and get 90 blocks weighted by Gaussian kernel on the spherical surface centered at x. Set a threshold intensity value T, then x can be encoded as a vector b composed of 90 elements with 0 or 1. 2. Get the maximal contiguous region of the spherical surface with a fast clustering method and count the number of blocks F ðxÞ in the

123

Int J CARS maximal contiguous region as the structural descriptor value of x. The structural similarity ufm(x) of If and Im at voxel x is defined by the Gaussian distance of their structural value in equation (1).    Ff ðxÞ  Fm ðxÞ ufm ðxÞ ¼ 1  a 1v ð1Þ qu Then we use the 3D-FAST scores as another channel in MI calculation.   FastMI If ðxÞ; Im ðTð/; xÞÞ; ufm ðxÞ; X  XX   pu if ; im ¼ pufm if ; im log  fm ð2Þ p if  pðim Þ im if   where pufm if ; im is calculated by equation (3) in which ufm(x) is calculated by 3D-FAST and k is a constant and can be adjusted by users. 1 X if  If ðxÞ im  Im ðTð/; xÞÞ pufm ðif ; im Þ ¼ k  ufm ðxÞvð Þvð Þ ð3Þ jXj x2X qp qp Finally, we use FastMI as similarity measure with a B-spline deformation field and optimize the cost function using the gradient descent algorithm [2]. Results We compared the performance of FastMI to global MI (gMI) in the presence of noise and inhomogeneity are shown in Fig. 1. For the first image pair (Fig. 1a, b), we moved the right one along the horizontal and the vertical axis and calculated the similarity value between the image pairs using FastMI and gMI. The results are plotted in Fig. 1e, f) for gMI and FastMI, respectively. The second image pair (Fig. 1c, d) get the results plotted in Fig. 1g, h). It can be observed that FastMI can still work when there are severe noise and inhomogeneity, but gMI fails.

Table 1 continued No

Inital

gMI

FastMI

5

2.42 ± 0.72

2.10 ± 0.73

1.62 ± 0.53

6

1.91 ± 0.56

1.86 ± 0.66

1.60 ± 0.54

7

3.85 ± 1.08

3.38 ± 0.97

2.56 ± 0.95

8

4.31 ± 1.22

3.76 ± 1.15

2.73 ± 1.10

Mean

3.99 ± 1.22

3.23 ± 1.12

2.63 ± 1.02

Max

6.01 ± 1.81

4.39 ± 1.54

3.79 ± 1.42

Conclusion In this paper, we extend traditional 2D FAST corner detector to a 3D structure descriptor, then we use this descriptor as spatial and geometric cues of CMI [3]. The method we proposed is demonstrated to be more robust and accurate than traditional registration method based on information theory. Acknowledge This study was supported by NSFC projects 81471758, 81271670 and MOST project 2015BAK31B01. References [1] Rivaz H, Zahra K, Fonov VS, Collins DL (2014) Nonrigid Registration of Ultrasound and MRI Using Contextual Conditioned Mutual Information. Medical Imaging, IEEE Transactions on 33(3):708–725. [2] Sotiras A, Davatzikos C, Paragios N (2013) Deformable Medical Image Registration: A Survey. Medical Imaging, IEEE Transactions on. 32:1153–90. [3] Loeckx D, Slagmolen P, Maes F, Vandermeulen D, Suetens P (2010) Nonrigid Image Registration Using Conditional Mutual Information. Medical Imaging, IEEE Transactions on 29 (1):19–29.

3D-printed beam modifiers for radiobiological experiments in monoenergetic carbon ion beams

Fig. 1 The performance of FastMI to global MI in presence of noise and inhomogeneity We also compared the accuracy of FastMI with gMI using images obtained from the BrainWeb database. Table 1 lists the mean target registration error of eight experiments with different initial deformation. FastMI shows higher accuracy than gMI under the same terminate conditions. Table 1 The accuracy of different registration methods No

Inital

gMI

FastMI

1

5.08 ± 1.64

4.27 ± 1.54

3.19 ± 1.31

2

5.02 ± 1.62

4.39 ± 1.53

3.77 ± 1.42

3

3.30 ± 1.14

2.65 ± 1.04

2.43 ± 0.99

4

6.01 ± 1.81

3.40 ± 1.24

3.10 ± 1.18

123

A. Solovev1,2, A. Chernukha1, U. Stepanova1,2, M. Troshina1,2, E. Beketov1,2, E. Koryakina1, A. Lychagin1, V. Fedorov1, V. Pikalov2, M. Kostin2, V. Kharlov3, S. Ulyanenko1 1 A. Tsyb Medical Radiological Research Centre - branch of the National Medical Research Radiological Centre of the Ministry of Health of the Russian Federation, Radiation biophysics, Obninsk, Russian Federation 2 State Research Center of the Russian Federation ‘‘Institute for High Energy Physics’’ of National Research Centre ‘‘Kurchatov Institute’’, Protvino, Russian Federation 3 Fablab ‘‘Model Spectr’’, Obninsk, Russian Federation Keywords Carbon ion radiotherapy  Monte-Carlo simulation  RBE  3D printing Purpose The most essential part implying future clinical use of carbon ion radiotherapy is the proper determining the dose deposit profile, doseweighted LET profile and the effective particle spectrum at the object. The first beam modifier was made as ripple filter [1,2] to provide a slight spreading near Bragg-peak region from single monoenergetic carbon beam for dose profile with high increasing of dose-weighted LET in this region in order to provide the similar dose in cell monolayers inside the flacon de Carrel with significant difference in LET values. The ongoing study now provided by A. Tsyb MRRC and IHEP are to investigate properties of such modifiers in wide variety of radiobiology studies and for the future use in radiotherapy application with carbon ion beams.

Int J CARS Methods The filter design and simulations of dose deposit, primary and secondary particles LET spectrum inside cell layers were performed using Geant4.10-1. The whole setup including 40 cm water phantom with 3 cm polycarbonate walls and small air or water-filled kesson placed inside water phantom with high precision movement device was taking into account in Geant4 models. Also the NPLibrary software presented by MRRC a previous year [3,4] coupled with CERN ROOT package and specially developed software in Python has been used for data analysis. The designed filter was made on ‘‘Dimension Elite’’ 3D-printer at fablab ‘‘Model Spectr’’. The physics experiments were made on U-70 synchrotron at IHEP, Protvino, Russia providing wide (6 cm) carbon ion beam with initial energy of 455 MeV/nucleon. The TM30011 ionization camera with Unidos was used for direct measurements, flat cameras with Mylar walls and LiF neutron detector, calibrated with Pu) was used for intensity and monitor units control. The Gafcromic EBT3 film was used both for uniformity control in transverse direction and for dose deposit control alongside. The cell cultures in monolayers were irradiated inside pair the flacon de Carrel placed face-to-face inside air kesson (see bottom left photo on Fig. 1). The experiment were done in April, 2015.

433 MeV/nucleon while green curve is simulation results with passive ripple filter depicted on the right. The black circles are preliminary measurements taken with this filter in December 2015. Thus the passive ripple filters can be used for more precise Bragg peak spreading which is essential for radiobiology studies and in future for the practical use in radiation treatment to form the proper biological effective dose distribution in the target. The later simulations performed on IHEP computation cluster (RU-Protvino-IHEP site, one of three biggest WLCG Tier-2 centers in Russia).

Fig. 2 Simulated 1D dose profile and LET profile, measured data and photo of new designed ripple filter

Fig. 1 Photo, simulated and measured Bragg curvers Results The simulation shows that the difference in material composition of ABS is not essential for dose deposit in object for the current initial beam energy. However, on practice with lower energies, this can be essential and the additional studies required. The resulted filter was chosen based on the number of simulations and was made with 40 groups, each group had as 5 sections with 356 um width and 6, 4, 2, 1.6, 1 mm height. The measurement (see Fig. 1) shows good agreement with the simulation. The pair of monolayers inside the flacon de Carrell assessed to receive similar (within 2 %) dose with significant change of dose-weighted LET (80 ± 5 and 160 ± 10 keV/um). The dose uniformity measured inside the water with transverse-placed EBT3 film near the end of SOBP is 97 %. The results of radiobiological studies made with this beam modifier will be presented later. Conclusion The most essential practical outcome of this study is that the actual radiobiological properties of carbon ion beam (especially, the RBELET dependence) can be properly investigated using monoenergetic beams. The main advantage of the using the ripple filters is that dose deposit profile is formed simultaneously from one monoenergetic beam while active SOBP require different beams and each can carry its uncertainty. On Fig. 2 below the black curve is the simulation curve formed with active beam technique with 7 beams from 420 to

References [1] U. Weber, G. Kraft Design and construction of a ripple filter for a smoothed depth dose distribution in conformal particle therapy//Phys. Med. Biol., No. 44, 1999. pp. 2765–2775. [2] T.P. Ringbaek, U. Weber, al. E. Monte Carlo simulations of new 2D ripple filters for particle therapy facilities//Acta Oncol., No. 53(1), 2014. pp. 40–49. [3] A.N. Solovev, U.A. Stepanova, S.E. Uliyanenko, A.E. Chernukha, V.V. Fedorov, Geant4-based framework for hadronic radiotherapy simulations//International Journal of Computer Assisted Radiology and Surgery, Volume 10 Supplement 1, June 2015, p. 201. [4] A. Solovev, A. Chernukha, U. Stepanova, V. Fedorov, Gent4-based hadron interaction optimization framework//Book of Abstracts/ Third International Conference on Radiation and Dosimetry in Various Fields of Research, RAD 2015, June 8–12, 2015, p. 314.

Combination of markerless surrogates for motion estimation in radiation therapy T. Geimer1,2, M. Unberath1,2, O. Taubmann1,2, C. Bert2,3, A. Maier1,2 1 Pattern Recognition Lab, Friedrich-Alexander-University ErlangenNuremberg, Germany 2 Graduate School in Advanced Optical Technologies, Erlangen, Germany 3 University Hospital Erlangen, Radiation Oncology, Erlangen, Germany Keywords Motion modeling  X-ray  Range imaging  Dimensionality reduction  Regression Purpose Respiratory motion drastically affects dose profiles in radiation therapy and needs to be compensated. Usually the internal motion is inferred from a correlated external surrogate [1, 2].

123

Int J CARS We propose an image-based model to estimate internal motion fields from X-ray fluoroscopy using dimensionality reduction and regression techniques. Further, we present results of an early study investigating possibilities to incorporate multiple surrogates, range imaging [3] and fluoroscopy, into the estimation process. Methods Recently, Taubmann et al. [3] proposed an approach to model dense deformation fields of both the internal organs and the external surface based on 3-D MRI sequences. Employing dimensionality reduction and multilinear regression the features of the internal motion model U ¼ ½/1 ; . . .; /n T 2 Rnl were estimated from the surface motion model features RRI ¼ ½rRI1 ; . . .; rRIn T 2 RnfRI , where n is the number of respiratory phases and l; fRI are the chosen feature space dimensionalities. Correlation between the features is understood as a multivariate multilinear regression (MLR) problem. Using 4-D CT data, we expand on this approach introducing fluoroscopy images as a surrogate. Digitally Reconstructed Radiographs (DRR) [4] are used for training purposes. Assuming that respiration is the main mode of variation among the images, the first few principal components of the set of all vectorized projection images are highly correlated to the breathing signal [5]. Principal Component Analysis (PCA) finally yields the feature matrix RFL ¼ ½rFL1 ; . . .; rFLn T 2 RnfFL , where fFL is the number of principal components for the fluoroscopy model. In some cases, multiple surrogates can be acquired during treatment. We consider possibilities of combining information from range imaging and fluoroscopy. The information added by the second surrogate may be used to improve the estimation or compensate for one surrogate failing. To this end, a combined low-dimensional feature vector of both surrogates is created:

 rRIi 2 RðfRI þfFL Þ1 : rCBi ¼ rFLi Then, the feature matrix RCB ¼ ½rCB1 ; . . .; rCBn T 2 RnðfRI þfFL Þ is used as the surrogate input for regression. The number of retrievable internal features is determined by the column rank r of RCB . If the feature vectors dRIi and dFIi are linearly independent, suggesting that they contain partially unique information, r  fRI þ fFL can exceed the single surrogate bounds. This observation indicates that it may be possible to correctly estimate multiple target features while only relying on a few surrogate features that are shown to correlate well (see Table 1). Table 1 Pearson’s correlation coefficient of the internal model features and the two surrogate features of the first three patient data sets Component

Pat1 1

Pat2 2

3

1

Pat3 2

3

1

2

3

jcorð/; rRI Þj

0.98 0.85 0.59 0.99 0.99 0.97 0.96 0.92 0.96

jcorð/; rFL Þj

1.0

0.99 0.57 0.99 0.91 0.88 1.0

jcorðrRI ; rFL Þj 0.97 0.85 0.15 1.0

0.45 0.39

0.94 0.91 0.96 0.36 0.20

The approach was evaluated on nine 4-D CT patient data sets consisting of ten volumes each. Registration provides nine deformation fields describing distinct motion states. Estimation accuracy was assessed in a leave-one-out study for each data set, where each phase was subsequently chosen as the test phase. We also excluded the two neighboring phases from training to prevent bias. The remaining six phases were used to train the correspondence models. Accuracy was defined as the root-mean-square error w.r.t. vector magnitudes between the estimated deformation field and the ground truth deformation field of the test phase.

123

Results Table 1 shows correlation results for three out of nine patients. Exemplary for l; fRI ; fFL ¼ 3 the first to third PCA scores were compared. While the first component correlates well with the internal model, the second one is varying significantly, with the third one mostly correlating poorly. Figure 1 shows the mean estimation error over nine data sets for the three approaches surface (RI), fluoroscopy (FL), and their combination (CB). Fluoroscopy outperformed the surface with the lowest error of 0:67 0:33 mm for l ¼ 2. However, accuracy did not improve with higher internal model dimension. Combining two surrogates in the proposed manner did not yield consistent improvement. For fRI ; fFL ¼ 1 both features represent the respiratory phase making them redundant. Thus, for a (1=1) combined feature vector the rank-deficient surrogate matrix is unable to explain two internal features. In contradiction to the other results, estimating l ¼ 2 internal features from a (2=2) or higher combined feature vector is promising with the best overall estimation of 0:62 0:28 mm, indicating that a combination of surrogates is useful under certain circumstances. The mean error without compensation was 2:3 0:70 mm.

Fig. 1 Mean error and standard deviation for estimation based on single and multiple surrogates over nine patient data sets. CBðx=xÞ denotes estimation of l internal features from a combination of x features of each surrogate (with l  3 being underdetermined for x ¼ 1) Conclusion The combination of surrogates did yield improvements, however they were only minor. This suggests that for future work more sophisticated approaches need to be explored in order to extract mutually exclusive information from the surrogates. Further, a detailed rank analysis of the regression matrix can help identify conditions in which a combined approach is useful. References [1] J R McClelland et al. (2013) Respiratory motion models: A review. Med Image Anal, 17(1):19–42. [2] M Wilms et al. (2014) Multivariate regression approaches for surrogate-based diffeomorphic estimation of respiratory motion in radiation therapy. Phys Med Biol, 64(5). [3] O Taubmann et al. (2014) Prediction of Respiration-Induced Internal 3-D Deformation Fields From Dense External 3-D Surface Motion. CARS 2014, pp. 33–34. [4] G W Sherouse et al. (1990) Computation of Digitally Reconstructed Radiographs for Use in Radiotherapy Treatment Design, Int J Radiat Oncol, 18(3):651–658. [5] P Fischer et al. (2014) Real-Time Respiratory Signal Extraction from X-ray Sequences using Incremental Manifold Learning. ISBI 2014, pp. 915–918.

Catheter targeting under electromagnetic guidance in breast brachytherapy: a demonstration of concept T. Vaughan1, H. Brastianos2, A. Lasso1, M. Westerland2, T. Ungi1, C. B. Falkson2, G. Fichtinger1 1 Queen’s University, School of Computing, Kingston, Ontario, Canada 2 Kingston General Hospital, Radiation Oncology, Kingston, Ontario, Canada

Int J CARS Keywords Breast  High dose rate brachytherapy  Electromagnetic guidance  Needle guide Purpose Breast cancer is the most common cancer in women in Canada with a life-time risk of 1 in 9 [1]. Many women are treated with lumpectomy followed by radiation therapy. Accelerated Partial Breast Irradiation with High Dose Rate (HDR) brachytherapy by means of interstitial catheters is a treatment option. Hollow needles are inserted into the breast and then a flexible catheter is passed through each needle. The needles are removed, leaving the catheters in the breast. The catheters need to be positioned in and around the tumour bed with even spacing. Fluoroscopy or ultrasound (US) is used to localize the tumour bed, but the breast is a highly deformable organ and it is difficult to maintain a sense of catheter placement relative to the tumour bed. To ensure correct catheter placement and spacing we propose to apply an electromagnetic (EM) guidance system. Methods An EM-tracked localization needle is inserted into the tumour bed under US guidance, and a hook is deployed to anchor the needle in place. This establishes a coordinate system locally-rigid to the tumour bed (Fig. 1A). The tumour bed is segmented in this coordinate system using tracked US, which creates a tracked model of the tumour bed in a virtual 3D view (or simply ‘‘view’’, Fig. 1B). In these aspects our approach derives from Ungi et al. [2].

catheter needle has been inserted, all planned catheter paths are drawn on the view to facilitate evenly-spaced, parallel catheter insertions. The guide is aligned with a planned catheter path on the view for each subsequent insertion. Our software is built on the 3D Slicer [3] (www.slicer.org) and PLUS [4] (www.plustoolkit.org) platforms. In the current implementation, US image and tracking data are collected by PLUS running on an Ultrasonix SonixTouch with GPS extension (Ultrasonix, Richmond, BC, Canada) and relayed to 3D Slicer on a navigation computer (Fig. 1A). Results A radiation oncology resident performed catheter needle insertions on breast phantoms made with soft plastic (M-F Manufacturing, Fort Worth, TX, USA) in two separate experiments. The phantoms had CT- and US-visible tumour beds within them. In all experiments, the goal was to insert catheters through the tumour bed with 1 cm spacing between catheters. Catheter insertions were carried out under US guidance or under combined EM-US guidance (as described above). CT scans of the phantoms were acquired after the insertions to verify the positions of the catheters with respect to the tumour bed. In the first experiment, one row of four catheters was inserted into each of four phantoms (two phantoms using US guidance, two phantoms using EM-US guidance). The phantoms were all placed on a flat table. Under US guidance only, seven of eight catheters total passed through the tumour bed. Under EM-US guidance, all eight catheters passed through the tumour bed. We did not observe improvement in maintaining even spacing between catheters in this experiment, though we hypothesize that the table acted as a parallel reference plane for the needle insertions. In the second experiment, two rows of five catheters were inserted into each of two phantoms (one phantom using US guidance, the other using EM-US guidance). The phantom was placed on a model of a human torso in order to more accurately simulate the procedure. Under US guidance only, seven of ten catheters passed through the tumour, and the catheters showed uneven spacing (Fig. 2A). Under EM-US guidance, nine of ten catheters passed through the tumour, and the catheters were inserted with uniform spacing (Fig. 2B).

Fig. 2 CT cross-sections of phantoms with inserted catheter needles. Tumour bed appears light, catheter needles appear as white spots. (A) US guidance. (B) EM-US guidance Fig. 1 Navigated HDR breast brachytherapy system setup. (A) Overview showing the components of the system used in this study. Blue coils represent EM sensors. (B) View shown from the position and orientation of the guide. (C) The rapid-prototyped guide used in this study The radiation oncologist holds a tracked needle guide (or simply ‘‘guide’’, Fig. 1C) against the breast and points it toward the tumour bed with the help of the view. When the guide is pointed at the tumour bed, a catheter needle is inserted though the tissue. After the first

Conclusion The results from our experiments demonstrate that EM guided HDR breast brachytherapy is feasible in phantoms. Further research is warranted, and work is already underway for the possible translation of our workflow to clinical use. EM guidance also offers other opportunities in HDR brachytherapy. Segmentation of the catheter paths on the planning CT need not be done manually but could be recorded by a small sensor as it is fed through each catheter. It is also possible to track the catheter needle path in real time using EM tracking [5]. Further research into these aspects is being pursued.

123

Int J CARS Acknowledgements: T Vaughan is funded by an Alexander Graham Bell Canada Graduate Doctoral Scholarship. G Fichtinger is funded as a Cancer Care Ontario Research Chair. References [1] Canadian Cancer Society’s Advisory Committee on Cancer Statistics (2015) Canadian Cancer Statistics 2015. Toronto, ON: Canadian Cancer Society. [2] Ungi T, Gauvin G, Lasso A, Yeo C, Pezeshki P, Vaughan T, Carter K, Rudan J, Engel C, Fichtinger G (2015) Navigated breast tumor excision using electromagnetically tracked ultrasound and surgical instruments. IEEE Trans Biomed Eng. 63(3):600–06. [3] Pieper S, Halle M, Kikinis R (2004) 3D Slicer. In IEEE Symposium Biomed Imaging. Apr 15 (pp. 632–635). [4] Lasso A, Heffter T, Rankin A, Pinter C, Ungi T, Fichtinger G (2014) PLUS: open-source toolkit for ultrasound-guided intervention systems. IEEE Trans Biomed Eng. 61(10):2527–37. [5] Lugez E, Sadjadi H, Joshi CP, Akl SG, Fichtinger G (2016) Improved electromagnetic tracking for catheter path reconstruction in high-dose-rate brachytherapy. Medical Physics. Accepted conditionally.

Development of an open PET system for image-guided surgery H. Tashima1, Y. Yoshii1, Y. Iwao1, E. Yoshida1, H. Takuwa1, H. Wakizaka1, T. Yamaya1 1 National Institute of Radiological Sciences, Chiba, Japan Keywords PET-guided surgery  OpenPET  Image-guided surgery  Surgery support system Purpose Tumors diagnosed as malignant are generally removed surgically. Conventionally, surgery supporting systems such as fluorescent or X-ray imaging have been used to improve the success rate. However, in cases where the tumors are widely and complexly distributed and they move with the organs or they are located on the backside of the organs, it is challenging to remove all the tumors in one surgery because it is difficult to image them due to low penetration of the light for the fluorescent imaging and low contrast of soft tissues for the X-ray imaging. To deal with this problem, we are aiming at developing a positron emission tomography (PET)-guided surgery system [1,2,3], (Fig. 1), in which we can perform surgery while confirming the three-dimensional positions and distributions of tumors with high sensitivity. Toward this goal, we are developing the world’s first open-geometry PET scanner, OpenPET, which can provide an accessible open space to the patient during PET scanning and realtime imaging system where the image reconstruction process, which typically takes more than several minutes, can be done in less than 1 s. The proposed system can provide real-time PET imaging during the surgery so that physicians can perform the surgical operation while confirming the tumor locations from the images. In this study, we demonstrated the concept of the real-time OpenPET-guided surgery by implementing the system with a small OpenPET prototype and by conducting actual surgery to remove cancer tumors from a mouse.

123

Fig. 1 Conceptual illustration of the real-time OpenPET-surgery Methods The small prototype used for the demonstration was based on the second generation of the OpenPET, a single-ring OpenPET (SROP), which has the shape of a cylinder cut by two slanted parallel planes to form an open space. In this prototype, block detectors originally forming a conventional cylindrical PET scanner are axially shifted little by little, in a manner we call axial-shift type SROP. The prototype has a detector ring with a diameter of 250 mm that includes 16 detector units each of which consists of two depth-of-interaction (DOI) detectors. The center of each detector surface, positioned on the parallel planes, is slanted 45 against the axial direction to have an open space of 139 mm. Each DOI detector consists of a 64-ch flat panel position sensitive photomultiplier tube, and the 4-layer 16 9 16 array of Zr-doped GSO scintillators with a size of 2.8 9 2.8 9 7.5 mm3. Axial length of the field of view (FOV) with a parallelogram shape is 102 mm. The spatial resolution average over the FOV is 2.6 mm in full width at half maximum. The sensitivity at the center of the FOV is 5.1 % and similar to that of commercial small animal PET scanners. For the surgery using the mouse, an operation table was set at the center of the FOV. The open space made it possible to perform the operation while the tumor was located inside the FOV of the OpenPET. For real-time image reconstruction, we implemented the 3D one-pass list-mode dynamic row-action maximum likelihood algorithm on the graphics processing unit. The system could display images with an arbitrary accumulation time frame in real-time; in other words, the images became clear gradually as the accumulation time increased. Human colon carcinoma HCT116-RFP cells had been intraperitoneally transplanted into a mouse. One hour after FLT (18Ffluorothymidine) injection of 3.7 MBq, the mouse was set onto the operation table inside the FOV of the OpenPET for an abdominal operation. Results Figure 2 shows the result of the surgical operation supported by the developed system. At first, we checked the tumor locations by the OpenPET imaging when the mouse was set on the operation stage. Measurement time required to acquire sufficient numbers of

Int J CARS data to visually identify the tumors from background radioactivity with clear contrast was about 20–30 s. The ratio of radioactivity concentration of the tumors to the background was 2–3 (tumor:background = 2–3:1). On the other hand, the radioactivity concentration of urinary bladder was 8 times higher than that of the tumors. Therefore, there was room for improvement such as using different tracers in the case where the tumors are located near the urinary bladder. Images were reconstructed with the computational time of less than 1 s, and displayed images became gradually clearer every second. After removing the tumors and placing them outside the body, we could confirm by PET images that the region with radioactivity concentration had been appropriately isolated.

[3]

Tashima H, Yoshida E, Kinouchi S, Nishikido F, Inadama N, Murayama H, Suga M, Haneishi H, Yamaya T (2012) RealTime Imaging System for the OpenPET, IEEE Trans. Nucl. Sci., vol. 59, no. 1, pp. 40–46.

Image reconstruction for surgical navigation using intraoperative PET-laparoscope M. Liyanaarachchi1, K. Shimazoe2, H. Takahashi2, E. Kobayashi3, I. Sakuma3 1 The University of Tokyo, Dept. of Bioengineering, Tokyo, Japan 2 The University of Tokyo, Dept. of Nuclear Engineering and Management, Tokyo, Japan 3 The University of Tokyo, Dept. of Precision Engineering, Tokyo, Japan Keywords PET  Intraoperative  Lymph node metastasis  Laparoscopic Purpose As the therapy for gastric cancer, laparoscopic assisted surgery is used to remove affected parts of the stomach, which include a primary tumor and lymph nodes metastasis. Positron Emission Tomography (PET) is used in preoperative evaluation of the surgery and it could localize lymph nodes metastasis. But it’s difficult to locate those during the actual surgery because of their location changes and this leads to unnecessary tissue removal. Although the primary objective of the surgery is fulfilled, excessive resection may result in deterioration of Quality of Life (QOL) of patients. Hence to increase the QOL, intra-operative identification of lymph nodes metastasis is necessary. PET-Laparoscope system utilizing positron emitters has been proposed [1] where an intraoperative image is obtained by two gamma ray detector systems; one fixed detector array connected to the surgical bed and a movable detector probe which is inserted into the patient’s stomach with the laparoscopic instrument, as shown in Fig. 1. The image was firstly acquired by simple back projection method however the resulted image was not optimized enough.

Fig. 2 Demonstration of the real-time OpenPET-surgery Conclusion We applied the OpenPET real-time imaging system for PET-image guided cancer extirpation surgery. The surgery demonstrated that the system allowed us to confirm tumor positions anytime during the operation. We concluded that the proposed system was effective in preventing any tumors, especially those located behind organs, from being left after the surgery. References [1] Yamaya T, Inaniwa T, Minohara S, Yoshida E, Inadama N, Nishikido F, Shibuya K, Lam CF, Murayama H (2008) A proposal of an open PET geometry, Phys. Med. Biol., vol. 53, no. 3, pp. 757–73. [2] Tashima H, Yamaya T, Yoshida E, Kinouchi S, Watanabe M, Tanaka E (2012) A single-ring OpenPET enabling PET imaging during radiotherapy, Phys. Med. Biol., vol. 57, no. 14, pp. 4705–18.

Fig. 1 Conceptual diagram of PET-laparoscope system The objective of this research is to improve the Signal-to-Noise Ratio (SNR) of PET-laparoscope system, which requires the image reconstruction using movable detectors. The image is reconstructed using Filtered Back Projection (FBP) algorithm which is a commonly used method in conventional PET systems. The conventional FBP algorithm for cone beam geometry is adjusted to incorporate application requirements. The position information acquired by a position tracker like Polaris Optical Tracking Systems (Northern Digital Inc., Ontario, Canada) is combined to the reconstruction system in the actual use. A suitable geometry for the system is investigated by simulation data. System should have spatial resolution of 10 mm according to medical requirements.

123

Int J CARS Methods The setup of detectors in PET-laparoscope system is simulated using Geant4 Monte-Carlo simulation platform (International Geant4 Collaboration). As the fixed detector system, a 16 9 8 array of 10 9 10 9 20 mm GAGG (Gd3Al2Ga3O12) crystals is used. The movable probe detector is a single 10 9 10 9 20 mm GAGG crystal. Fluorine-18 (18F-FDG) point source is used to represent a cancerous lymph node. 120 9 106 events are used to simulate a 2 min scan for reconstructing a 1 MBq source. The distance from fixed detector to source is set to 12 cm considering human anatomy. Distance from movable detector to source is varying and is about 2 cm. The annihilating gamma rays emitted by the source are detected and the coincidence detection of gamma rays is used to reconstruct the image in an image reconstruction program written in C++ programing language. The geometry of detectors in PET-Laparoscope system has several differences to the conventional cone beam geometry. Fan beams are not symmetric because the detector probe is the only moving component. The moving path is not circular and movements do not happen in the same plane. There is low angular coverage due to constraints in probe moving during laparoscopic surgery. To incorporate those factors, the FBP for conventional cone beam geometry [2] is adjusted and modified accordingly. For the reconstruction, an asymmetric fan beam is considered as a part of a symmetric fan beam for an imaginary detector array. Incorporation of the random movements of the movable detector is done similar to the generalized cone beam algorithm [3]. Results Simulation setup built in the Geant4 platform and the reconstructed image obtained by moving the detector probe in an area covering a Field of View (FOV) of 400 is shown in Fig. 2a, b. As shown in Fig. 2c the proposed method could identify two radiation sources in 10 mm distance. The obtained FWHM (Full Width at Half Maximum) image resolution is 4.4 mm in X direction, 5.6 mm in Z direction and 14 mm in Y direction. 5.4 mm location shift in Y direction is observed. Low resolution and location shift in Y direction are due to the low angular coverage of the Volume of Interest.

Fig. 2 (a) Simulated experimental setup (b) The reconstructed image for one source (c) The reconstructed image for two sources with 10 mm distance Simulation studies show that to obtain the required resolution and to eliminate the shift in Y direction, angular coverage of at least 1200 is required. But in current detector setup, when the movable detector is rotated further coincidence count is reduced since there are no enough crystals in opposite direction to capture gamma rays. Hence the fixed detector have to be modified so that it covers a larger region. Conclusion The point source was successfully reconstructed and identified from the reconstructed image in PET-laparoscope system. The depth wise spatial resolution is relatively worse with FBP algorithm due to the low angular coverage of measurements. By widening the fixed detector and applying sensitivity correction to remove the geometrycal dependencay of sensitivity, better results are expected. Acknowledgement This study is partly supported by the Grant for Translational Systems Biology and Medicine Initiative from the Ministry of Education, Culture, Sports, Science and Technology of Japan.

123

References [1] Shimazoe K, Takahashi H, Sakuma I, Takahashi M, Momose T (2015) Prototype of a single probe PET laparoscope system. Int J CARS 10(Suppl 1): S115. [2] Kak AC, Slaney M (1988) Principles of Computerized Tomographic Imaging. IEEE Press. [3] Wang G, Lin TH, Cheng P, Shinozaki DM (1993) A General Cone-Beam Reconstruction Algorithm. IEEE Transactions on Medical Imaging 12(3): 486–496.

Clinical impact of quantitative post-radioembolization 90Y PET/CT using next-generation digital photon counting PET detectors C. Wright1, K. Binzel1, J. Zhang1, E. Wuthrick2, C.- H. Tung3, M. Knopp1 1 The Ohio State University, Wright Center of Innovation, Department of Radiology, Columbus, United States 2 The Ohio State University Wexner Medical Center, Department of Radiation Oncology, Columbus, United States 3 Philips Healthcare, Cleveland, United States Keywords Digital PET/CT  Radioembolization  90Y PET/CT  Theranostics Purpose At present, pre-radioembolization therapy planning is performed using intrahepatic arterial administration of 99mTc macroaggregated albumin (MAA) and then SPECT/CT is used to assess the intrahepatic MAA biodistribution and exclude any significant non-targeted/extrahepatic shunting. Post-radioembolization assessment of intrahepatic 90Y activity is commonly performed with bremsstrahlung SPECT/CT but this approach is limited by poor image quality and lack of accurate quantification due to extensive scatter radiation. Previously we have demonstrated that it is clinically feasible to image Yttrium-90 radioactivity using next-generation digital photon counting PET detector (dPET) technology following interventional radioembolization in patients with hepatic malignancies and metastases. dPET offers greater count sensitivity as well as an adjusted energy window specific for 90Y imaging, making it practical to image such a low abundance isotope. The purpose of this study is to qualitatively and quantitatively assess 90Y biodistribution following radioembolization using dPET/CT and compare to pre-/post-radioembolization SPECT/CT approaches. Methods In an ongoing clinical trial, pre-radioembolization MAA SPECT/CT and post-radioembolization bremsstrahlung 90Y SPECT/CT imaging was performed (Symbia, Siemens) in 15 patients who were treated with 90Y-labeled glass microspheres for malignant/metastatic hepatic lesions. 90Y time-of-flight digital PET/CT (Vereos, Philips) was subsequently performed in each patient within 3 days following radioembolization. In this study, dPET/CT imaging was performed using a faster total image acquisition time (21 min) than SPECT/CT (22 min) for the same imaging volume. Image quality of SPECT/CT and dPET/CT was evaluated, by matched pair comparison, as well as the distribution of intrahepatic radioactivity on pre-/post-radioembolization imaging. Pre-/post-radioembolization concordance and volumetric assessment of intrahepatic radioactivity was also evaluated using a semi-automated post-processing software. Results In this study, all patients had evaluable SPECT/CT, bremsstrahlung SPECT/CT and dPET/CT images for the qualitative assessment of intrahepatic radioactivity and imaging concordance/discordance for pre-/post-radioembolization imaging. Consistently, dPET/CT imaging of 90Y activity yields better image quality with improved 90Y-tobackground contrast than standard bremsstrahlung SPECT/CT

Int J CARS imaging. There is also improved assessment of the intrahepatic biodistribution of 90Y radioactivity with dPET which enables better evaluation of imaging concordance/discordance with pre-radioembolization MAA SPECT/CT, see also Fig. 1.

Fig. 1 Shows fused axial images of pre-radioembolization 99mTcMAA SPECT/CT (top left), post-radioembolization bremsstrahlung SPECT/CT (top middle) and dPET/CT (top right) and the same axial fused images with threshold-based isocontours (bottom) for quantitative volumetric assessment of intrahepatic radioactivity Conclusion These findings demonstrate that next-generation digital photon counting PET detector technology allows for faster imaging of intrahepatic 90Y activity with better image quality, 90Y-to background contrast and quantitative accuracy than existing bremsstrahlung SPECT/CT approaches. Given its faster image acquisition time, digital PET/CT can address the current unmet clinical need for more efficient and accurate imaging based assessment of 90Y following interventional radioembolization. Furthermore, dPET/CT enables more precise assessment for imaging concordance/discordance of 90Y microsphere deposition within the liver when compared with pre-radioembolization MAA SPECT/CT. This digital PET detector technology will likely create new computerassisted quantitative methodologies for dosimetry that will also benefit patients treated with new and existing 90Y-based theranostics.

Kalman filter based sensor fusion for needle tracking in MR-guided cryoablation W. Gao1, D. Kacher2, B. Feltics3, E. Nevo3, T. Lee2, J. Jayender2 1 Harbin Institute of Technology, School of Life Science and Technology, Harbin, China 2 Brigham and Women’s Hospital, Harvard Medical School, Department of Radiology, Boston, United States 3 Robin Medical Inc., Baltimore, United States Keywords Kalman filter  Sensor fusion  Needle tracking  MRI-guided cryoablation Purpose Magnetic resonance imaging (MRI)-guided tumor cryoablation is a promising procedure due to its high success rate and low complications. However, due to the limited size of MRI gantry, the patient has to be repeatedly moved in-and-out of the gantry for scanning to verify the needle trajectory. This could be time-consuming and fatiguing to both the patients and clinicians. Real-time tracking of the needle fused with

intraoperative MRI is vital to accurately navigate the cryotherapy needle to the desired location, reduce intraoperative time and prevent damage to critical structures. We have used electromagnetic (EM) and optical tracking methods to track the needle tip in real-time. EM tracking has the advantage of tracking the needle tip in limited space without line-of-sight issues. However, it suffers from poor accuracy due to gradient field inhomogeneity. Optical tracking, on the other hand, has significantly higher accuracy than EM tracking but has a line-of-sight issue, which precludes its use for interventional applications. The purpose of this abstract is to develop an algorithm to fuse optical and EM tracking data and harness the benefits of both the tracking modalities. Methods We utilized a MRI-safe, 6 degree-of-freedom EM sensor (from Robin Medical Inc., USA) to track the needle in real-time. The EM sensor uses the gradient fields of the MRI to track its position and orientation. The sensor was attached to the shaft of the needle at 100 mm from the needle’s tip. The needle was also instrumented with an optical sensor to compensate for the gradient nonlinearities at the gantry’s entrance. A rigid body defined by four optical reflectors was fixed at the needle’s handle. The offset of the rigid body from the needle’s tip was determined by pivot calibration. Spatial transformation: The measurements of the needle by different sensors were converted into a common coordinate system of MR images. To obtain a consistent point set, an MRI compatible spine phantom with 7 fiducial markers was used. T1-weighted MR images (Axial VIBE sequence, image spacing: 1.023 9 1.023 9 1.5 mm3) were obtained and imported into 3D Slicer. The position of the fiducial markers, denoted as PImg, was labeled manually on the images. The phantom was then moved from the isocenter to the entrance by 650 mm where needle tracking was performed. The coordinates of the fiducials in the optical coordinate system, POpt, were obtained. The transformation ImgTOpt between the coordinate systems of optical device and MR images was estimated by minimizing the Euclidean distance of two point sets using least-squares method, i.e., ImgTOpt = argminTR||PImg-TPOpt||. The coordinate of the needle’s tip measured by the EM sensor (PEM) was converted into the coordinate system PImg as follows: PImg = PEM + 100r + d, where r is the orientation vector measured by EM sensor parallel to the needle’s axis, and 100 mm is the offset from the needle’s tip to EM sensor; d is the displacement vector of the phantom from the isocenter, i.e., d = [0 0 650]T. Due to the inhomogeneous magnetic field at the entrance, there is a significant bias error in EM sensor measurements. To estimate the bias error, the needle was moved in the x–y plane at the entrance of the MRI bore by freehand. Both EM and optical tracking data were collected. The optical data was assumed to be the gold standard and the bias error between EM data and the optical data in the image coordinate system was calculated by Img TOptPOpt-PEM E(PEM Img ) = Img . A uniform error map was generated using tri-linear interpolation. Sensor fusion: A Kalman filter (KF) based sensor fusion algorithm was developed to improve the accuracy of EM tracking. The state vector, x, is composed of position and velocity of the needle’s tip. The speed of needle insertion was assumed to be low and constant at each time interval between two samples. KF consists of two recursive steps: prediction and correction. For each measurement of EM sensor, these two steps were executed. (1)

Prediction •Estimate the prior state xk|k-1: xk|k-1 = Axk-1|k-1 + BE(xk-1|k-1) + wk, where E is the error map of EM sensor. •Compute the prior error covariance matrix pk|k-1: Pk|k-1 = APkT 1|k-1A + Qk-1

(2)

Correction •Compute the Kalman gain Kk: Kk = Pk|k-1HT (HPk|k-1HT + Rk)-1 •Estimate the posterior state xk|k: xk|k = xk|k-1 + Kk(zk-(Hxk|k-1 + vk))

123

Int J CARS •Estimate the posterior error covariance matrix Pk: Pk|k = (I-KkH)

 Pk|k-1 I dtI Here, A ¼ , B = [I 0]T, H = [I 0], dt is the time 0 I interval between two samples. The vectors, w and v, are the processing and measurement noise, respectively. The former one was set experimentally, and the latter one was measured during the calibration. Q and R are the corresponding covariance matrices. Results Figure 1 illustrates the experimental setup. The sub-figure shows the EM and optical sensors integrated with the cryoablation needle. The users performed needle insertion 4 times. Figure 2 shows the results of the experiments. The figure visually indicates that the data fusion method performs significantly better than EM tracking only. The statistics of the tracking error of the needle’s tip for the EM sensor and data fusion methods are summarized in Table 1. The mean position error with EM sensor only is higher than that obtained by KF-based data fusion algorithm.

the KF-based data fusion algorithm provides significantly better accuracy in tracking the needle’s tip than just the EM sensor. This can also be utilized to overcome the line-of-sight limitations of the optical sensor. The proposed method shows promise in facilitating accurate needle tracking in a constrained space to perform tumor cryoablation. Acknowledgement Chinese Scholar Council, National Natural Science Foundation of China (Grants No.81201150, 8150051096), Self-Planned Task (No. SKLRS201407B) of State Key Laboratory of Robotics and System (HIT), National Institutes of Health (Grant Numbers P41EB015898 and P41RR019703).

Ontology-based surgical process modeling by using SNOMED CT concepts and concept model attributes J. Neumann1, E. Schreiber1, T. Neumuth1 1 Leipzig University, ICCAS, Leipzig, Germany Keywords Surgical process modeling  Ontology  SNOMED CT  Workflow management

Fig. 1 Setup of experiments

Fig. 2 Results after space transformation and sensor fusion. BlueEM data after transformation, green-EM data after sensor fusion, redoptical data Conclusion Table 1 Error of needle tracking after space transformation and sensor fusion (unit:mm) Case

Space transformation

Sensor fusion

Mean

STD

Max

Mean

STD

Max

#1

3.18

1.26

#2

4.50

1.35

6.19

1.09

0.36

1.92

10.73

0.80

0.44

#3 #4

4.39 8.76

0.88 1.41

6.17 11.20

2.18

1.02 0.88

0.55 0.49

2.73 2.36

This paper proposes a method of Kalman filter-based sensor fusion for tracking the needle in MRI with two complementary techniques, i.e., optical tracking and EM tracking. Experimental results indicate that

123

Purpose Surgical workplaces are characterized by an increasing technical complexity and an immense amount of information. Future operating rooms need efficiency-oriented processes by providing workflow management and knowledge-based decision support. Therefore, the operating room processes and surgical activities must be described in a machine-readable format as Surgical Process Models (SPM) [1]. In literature, different approaches for formal ontological representations of SPMs were described [2, 3]. However, the authors used proprietary ontologies and terminologies. As a result, there exists currently no approach for ontology-based SPMs with an adequate amount of concepts for process modeling across surgical disciplines and intervention types. Hence, the adaption of these ontologies for new intervention types or changing workflows is a complex and timeconsuming process. The aim of this paper is to present a new approach for ontologybased surgical process modeling by using standardized medical ontologies. For this purpose, the widely used and clinically validated ontology, SNOMED CT, is utilized. Based on SNOMED concepts and concept model attributes, a generic approach for semantic surgical process modeling should be developed. The formal specification of surgical processes with a standardized ontology is a necessary requirement for different applications in modern operating room environments, e.g. workflow and information management or knowledge-based decision support. Methods For the development of a generic concept for surgical process models, 21 neurosurgical workflows have been chosen for analysis. The workflows have been recorded at the University Hospital Leipzig, using a formal approach for description and a surgical workflow editor for the acquisition of work steps [4]. First off, the relevant process elements and their relationships were extracted from the recorded workflows and analyzed for semantic workflow modeling. The surgical process elements were transferred into SNOMED concepts. Their relations were mapped to SNOMED concept model attributes, considering semantical constraints of SNOMED attributes, like ‘‘domain’’ and ‘‘range concept’’ restrictions. For the retrieval of the appropriate concepts and attributes the IHTSDO SNOMED CT Browser (International Edition, July 2015) was used. In the next step the generic concept has been evaluated in a proof-of-concept study for a microscopic lumbar discectomy workflow. Results Neumuth et al. proposed a 5-tupel for representing a complete surgical activity, containing the surgical action, the actuator and the body

Int J CARS part the actuator performed the action with, the used instruments and the treated anatomic structure [1]. In addition, the intervention type and the surgical phase were considered. These surgical process elements are represented in Fig. 1 with their semantic relations. Overall 39 relations were identified. By filtering bidirectional relations, 24 relations were presumed to be relevant for semantic surgical process modeling. However, only 13 relations were required for a semantical complete modeling of surgical processes. The remaining relations could be received by logical reasoning.

Fig. 2 (a) SNOMED concepts and attributes for surgical process elements and their semantic relations. (b) Example Activity: Excision of lamina of lumbar vertebra

Fig. 1 Surgical Process Elements with relevant semantic relations (black relations are required for complete modeling) In the following the surgical process elements and semantic relations were mapped to SNOMED concepts and concept model attributes. In Fig. 2a the surgical elements and surgical relations were presented with a SNOMED name and identifier. A successful mapping in SNOMED top-level concept definition has already been shown in [5]. When mapping the required semantic relations to SNOMED attributes, the inherent ‘‘domain’’ and ‘‘range’’ constraints for every attribute must be considered, since only specific top-level concepts could be used. For instance, ‘‘Activity (procedure)’’ has ‘‘Procedure device (attribute)’’ ‘‘Instrument (Device)’’. The domain of ‘‘Procedure device’’ is defined as concept ‘‘Procedure’’ and the range as ‘‘Device’’ and all its hierarchically descendants. By using the SNOMED inherent ‘‘is-a’’-relations, different granularity levels for procedure concepts (invention, phase, activity) as well as anatomic structures and instruments could be achieved.

In conclusion, for the complete ontology-based surgical process modeling every required surgical element and semantic relation could be transferred into SNOMED CT concepts and attributes. The generic concept was evaluated in a proof-of-concept study for a discectomy workflow (Fig. 2b). For practical process modeling often different relations of one type are needed, meaning there could be one or more instruments or different next and previous activities/phases related to one activity. Otherwise, there is only a 1:1-relationship for the actual activity, phase, intervention, anatomical structure and the actuator. In addition, there are different optional relations in specific modeling, since not every activity has a previous (start) or next activity (end), or an anatomic structure. Besides the defined surgical process elements in [1], SNOMED provides a tremendous amount of additional relations. Specific attributes could be used for more detailed surgical process modeling, e.g. ‘‘Process morphology’’ for defining abnormal anatomical structures or ‘‘Procedure substance’’ for medication during surgery. Conclusion In this paper a generic concept for ontology-based surgical process modeling using a standardized medical ontology was presented. For this purpose, the required surgical elements and relations were identified and could be transferred into SNOMED concepts and attributes. Based on these concepts a standardized and reusable representation of surgical processes and process knowledge with a consistent terminology could be achieved. The future work consists of the analysis which semantic relations are mandatory for logical reasoning in case of incomplete information availability in surgical process elements. Ontological reasoning

123

Int J CARS provides a basis for different applications, like recognition of surgical activities and phases in the context of workflow and situation awareness. Further applications may be functionalities in workflow management support, surgical knowledge representation and decision support. References [1] Neumuth T, Strauß G, Meixensberger J, Lemke H U, Burgert O (2006) Acquisition of Process Descriptions from Surgical Interventions. Database and Expert Systems Applications, Springer Berlin Heidelberg, 602–611. [2] Neumuth D, Loebe F, Herre H, Neumuth T (2011) Modeling surgical processes: a four-level translational approach. Artif. Intell. Med 51 (3): 147–161. [3] Katic´ D, Julliard C, Wekerle A L, Kenngott H, Mu¨ller-Stich B P, Dillmann R, Speidel S, Jannin P, Gibaud B (2015) LapOntoSPM: an ontology for laparoscopic surgeries and its application to surgical phase recognition. Int. J. CARS 10 (9):1427–1434. [4] Neumuth T, Durstewitz N, Fischer M, Strauß G, Dietz A, Meixensberger J, Jannin P, Cleary K, Lemke H U, Burgert O (2006) Structured recording of intraoperative surgical workflows. Proc. SPIE 6145, Medical Imaging 2006: PACS and Imaging Informatics 6145, 61450A-61450A-12. [5] Neumann J, Neumuth T (2015) Standardized Semantic Workflow Modeling in the Surgical Domain—Proof-of-concept Analysis and Evaluation for a Neurosurgical Use-Case. IEEE 17th Healthcom, iOR: 6–11.

Real-time image processing and analysis over an IP based video network for the operating theatre E. Bellon1, M. Sweertvaegher1, K. Schoonjans2, A. Fannes3, B. Koninckx3, N. Hermans1, T. Koninckx3, B. Van den Bosch1 1 University Hospitals Leuven, IT, Leuven, Belgium 2 University Hospitals Leuven, Medical Instrumentation, Leuven, Belgium 3 eSATURNUS, Leuven, Belgium Keywords Digital operating room  Real-time image processing  Real-time image analysis  Evaluation in surgical routine Purpose Our university hospital has a few years of experience with a commercial digital OR solution installed in 34operating rooms. Characteristic of this system is that video is converted right at the source into a stream of bytes and handled in digital formuntil its presentation on any screen in the room. All data transport is over a standard Ethernet network. This technology opens up possibilities for real-time image processing using the remote computing power available in the central data room of the hospital. One purpose of this presentation is to illustrate this potential using two processing applications that have been developed. A second purpose is to report on the results of the validation of these applications in daily surgical routine (a validation that has just started and is expected to be concluded before the conference).

123

Methods With this system (NUCLeUS by eSATURNUS) the video output from laparoscopic devices, monitoring equipment, imaging equipment, computers,… is immediately converted to an IP data feed on a conventional Ethernet network of 1 Gbit/s. Receivers on the backof the displays pick up the stream(s) they want to present. A Gbit/s uplink to the main equipment room enables central recording or recompression for broadcasting. HD video is supported by high quality compression in the transmission modules, using methods that guarantee extremely low latency (in the order of a few milliseconds). The receiver module can process up to four image streams simultaneously (using Field Programmable Gate Arrays or an embedded CPU as a coprocessor). It additionally can present an overlay on top of the real-time video. This overlay can contain simple status information, but can also serve to render enhanced reality data on top of the surgical video. Results Two real-time image processing applications have been developed sofar. The first one performs real-time image based estimation and prediction of the angular rotation of the endoscopic camera, and corrects the image on display for this rotation. In other words, the ‘horizon’ stay horizontal at all times irrespective of the orientation of the camera or the instruments. This is meant to facilitate surgery with angled endoscopes and to improve hand-eye coordination. The second application uses pattern recognition to identify and extract vascular structures and highlights those in the overlay on the live video. Here the aim is, e.g., to indicate areas with an unusually high density of superficual vascularisation. Due to the complexity of the algorithm, the frame rate is reduced and the latency is slightly increased. Therefore, these enhanced images are offered as an additional navigation image adjacent to the original image. Nearly all surgeons at the University Hospitals Leuven are using the system in daily routine (so far without image processing in the loop), many for several years already. The cardiac and vascular surgeons were the last to have their rooms equipped with this system about a year and a half ago. They don’t only appreciate the ease of recording interventions, but also the fact that the different cooperating teams (surgeons, nurses, anesthesiologist and potentially the perfusionist) can better coordinate their actions because they now see information that traditionally was not easily available. Evaluation of the image processing applications is just starting. Evaluation methodology is to demonstrate and train the surgeons and make the processing modules available for use at their discretion (albeit with follow up whether the system is used or why it isn’t, and with slight encouragement for testing it if necessary). All disciplines have access to the processing applications in parallel. Impressions are obtained from the surgeons informally, by day to day contact with them in the operating theatre or at staff meetings, and this for a duration of about 5 months. Conclusion Whether the real-time image processing applications described in this paper will increase surgical efficiency or quality has still to be investigated further. But this already illustrates that the architecture of a system in which video streams are handled in digital form can add a new dimension to the digital operating room, moving in a natural way from networking of devices towards including data analysis and realtime image processing within the infrastructure.

Int J CARS

20th Annual Conference of the International Society for Computer Aided Surgery President: Pierre Jannin, PhD (F)

123

Int J CARS Towards an intra-operative calibration-preserving freehand ultrasound system D. Dall’Alba1, F. Bovo2, D. Zerbato2, A. Forgione3,4 and P. Fiorini1 1 University of Verona, Department of Computer Science, Verona, Italy 2 B.B.Z. s.r.l., Verona, Italy 3 AIMS - Advanced International Mininvasive Surgery Academy, Milano, Italy 4 Niguarda Ca’ Granda Hospital, General Oncologic Minimal Invasive Surgery, Milano, Italy Keywords Freehand ultrasound calibration  Multiple depth geometric calibration  Magnetic fixation system  Intra-operative navigation Purpose Ultrasound (US) is a very interesting image modality to guide minimally invasive surgical procedures since it provides real-time image feedback to the surgeon. US do not use ionizing radiation and the required hardware is compact and light-weight; these characteristics ease the adoption of this modality in the operating room (OR) [1]. To further improve the guidance capabilities during surgery the US device can be integrated with a tracking system that measures the position and the orientation of the probe and tools during the acquisition [2]. Tracked US systems are also called freehand US systems (FUSs) to distinguish them from hardware 3D US probes. However, the use of FUSs in the OR is still limited by different factors that impose an intra-operative calibration step. In this work, we propose a solution for one of these factors: the presence of the protective sleeve around the probe (Fig. 1).

sleeve; the frame can be made of plastic for disposable use or of metal for sterilization. Results The experiments have been performed with an Ultrasonix MDP US device and a Claronav Micron Tracker H60 tracking system. The two components of the proposed magnetic fixation system have been manufactured with a rapid prototyping system Stratasys 3D uPrint. Each part includes 4 neodymium magnets, with a magnetization grade N45 and dimensions 10 9 593 mm. We performed the calibration with a linear probe with acquisition frequency of 10 MHz and a depth setting of 90 mm. We used a general purpose protective vinyl cover. The calibration phantom and the calibration method used in the study are based on the PLUS toolkit [3]. For the evaluation of the proposed system we considered 2 experimental conditions: detachment and fixed marker configurations. In the first condition we removed and put back the marker from the collar after each trial, in the second we always kept the marker and collar mounted in a fixed configuration. Both experimental conditions are evaluated with and without the presence of the protective plastic cover. When the collar and the marker were detached also the protective cover (if present) was removed from the probe. We performed 10 independent calibrations for each of the considered experimental conditions following the protocol described in [4]. Figure 2 present the mean accuracy and precision obtained.

Fig. 2 Plots of the accuracy and precision results for the linear probe

Fig. 1 From left to right: a 3D model of the proposed magnetic fixation system with the two parts detached; the system with the collar attached to the probe and the corresponding marker; the system assembled with the protective sterile sleeve Methods The design of the fixation device followed some observations and requirements. First of all, target US probes are manually operated: their ergonomics is thus important and must be preserved. Any device entering the operating area must be sterile or must be wrapped in a sterile sleeve. Using a sterile sleeve makes the device easier to manufacture as it does not need to be sterilizable. Unfortunately, it is not possible to insert optical tracker markers inside the sleeve without compromising the accuracy in pose measurements; therefore this option has been discarded in favor of external disposable or sterilizable markers. The proposed solution is to cover the probe and a coupling device with a disposable bag and to leave outside the markers with a complementary coupling device. The coupling devices use complementary pattern of magnets and shapes to ensure the correct matching. The setup is thus composed of two parts (visible in Fig. 1): a collar shaped to be firmly mounted on the probe and a frame that is coupled to the collar and carries the markers on adjustable stands. The collar is thin and lightweight, so it does not limit the usage of the probe nor it interferes with the insertion of the probe into the sterile

123

Statistical analysis based on the Student’s 2-tails T-Test, confirms that there is no significant difference (p \ 0.002) in terms of accuracy and precision for repeated calibration with and without the probe cover when the tracker sensor is kept in a fixed configuration. This fact confirms that the presence of the protective cover does not influence the calibration results and that, once attached, the two parts are firmly held by the fixation system. When markers are reattached during the tests, the difference is statistically significant (p \ 0.005) and better results are obtained when the sterile sleeve is present. This result can be explained by noticing that the coupling device has been designed to cope with the thickness of the protective plastic cover. Thus the coupling is more stable (firmer coupling) and more repeatable (less backlashes) when the cover is present. In a real scenario these differences are not meaningful, as they are in the order of 0.2 mm: well below the precision of the tracker. Conclusion In this work we have described a magnetic fixation system for FUS, which eases the fixation of an optical tracking system sensor in real OR conditions. The experimental results confirm that the proposed system allows performing the calibration of FUS outside the OR and using the calibrated FUS without significant modification of the standard protocol. Future developments of the system will focus on the improvement of the hardware design to support even more type of probes and to improve the accuracy and precision evaluation of the system.

Int J CARS References [1] Cleary K, Peters TM (2010) Image-guided interventions: technology review and clinical applications. Annu Rev Biomed Eng 12:119–142. [2] Treece GM, Gee AH, Prager RW, Cash CJC, Berman LH (2003) High-definition freehand 3-D ultrasound. Ultrasound Med Biol 29(4):529–546. [3] Lasso A, Heffter T, Rankin A, Pinter C, Ungi T, Fichtinger G (2014) PLUS: open- source toolkit for US-guided intervention systems. IEEE Trans Biomed Eng 61(10):2527–2537. [4] Mercier L, Lang T, Lindseth F, Collins LD (2005) A review of calibration techniques for freehand 3-D ultrasound systems. Ultrasound in Medicine & Biology 31(2):143–165.

also pre-calculated in an FEM analysis. The deformation is within the yield strain of the material.

Laparoscopic ultrasound manipulator using a spring based elastic mechanism J. Arata1,2, K. Fukami3, S. Oguri1, T. Ikeda1, R. Nakadate1, S. Onogi1, M. Sakaguchi3, T. Akahoshi1, K. Harada2, M. Mitsuishi2, M. Hashizume1 1 Kyushu University, Fukuoka, Japan 2 The University of Tokyo, Tokyo, Japan 3 Nagoya Institute of Technology, Nagoya, Japan

Fig. 1 Ultrasound manipulator was implemented as a hand-held device with a four-button handle that the surgeon directly grasps to send a motion command to the tip mechanism

Keywords Surgical robot  Manipulator  Laparoscopic ultrasound scan  Laparoscopy Purpose Laparoscopic surgery requires surgeons to have high technical skills because of the limited movements of laparoscopic surgical instruments inserted through trocars into the abdominal wall. Image guidance is one of the key technologies that can improve the surgical outcome in laparoscopy [1]. However, due to the large deformation of digestive organs, a computer-aided navigation system based on the pre-operative data cannot indicate the correct target position of a lesion (e.g., liver tumor and vessels invisible from the surface). Therefore, intra-operative and real-time image acquisition in laparoscopy would increase the precision by accurately indicating the location of a lesion hidden under the surface of organs. We thus developed a laparoscopic ultrasound (US) scan manipulator that can perform dexterous 2 degree-of-freedom (DOF) motion. As a major improvement of our precedent development [2], the manipulator presented in the paper consists of an elastic structure using springs thus enables safe US scan avoiding excess force applied to the inspecting organs. Methods The ultrasound manipulator was implemented as a hand-held device with a four-button handle that the surgeon directly grasps to send a motion command to the tip mechanism (Fig. 1). A newly developed spring-based elastic mechanism at the tip can perform 2 DOF motion (pitch and yaw, as shown in Fig. 2). The two flat springs (right (R) and left (L)) are separately actuated by servomotors along the long axis. The flat spring largely deform when transmitting and transforming the linear motion from the servomotors to the 2 DOF rotational motions at the tip. The movement of the two flat springs in the same direction is converted to pitch motion, and differential movement is converted to yaw motion. The surgeon can manually realize rotation along the long axis by rotating the handle along this axis. The flat springs (0.2 mm in thickness) were fabricated from Ni– Ti alloy to allow a high degree of deformation. Note that the deformation of the two flat springs is guided by two mechanical hinges attached at the tip on each DOF, and thus each spring adequately deforms, as pre-calculated in an FEM analysis. The working range is from 20 to 90 degrees in pitch and from +90 to -90 degrees in yaw, as

Fig. 2 A newly developed spring-based elastic mechanism at the tip can perform 2 DOF motion (pitch and yaw). The movement of the two flat springs in the same direction is converted to pitch motion, and differential movement is converted to yaw motion In the prototype implementation, a probe approximately 12 mm in diameter (L43 K, Hitachi Aloka Medical, Ltd., Japan) was used. The diameter of the manipulator is 15 mm and thus can be inserted through a 15 mm trocar. The thickness of the robot is 3 mm at the thinnest part. The manipulator is a^- mm in height, and 56 mm in thickness. For sterilization, the sheath is detachable from the handle by a snap-on mechanism. Results As a preliminary evaluation, we conducted a phantom scan experiment in a dry lab environment and an in vivo experiment. The dry lab experiment was conducted using a laparoscopy training box. A phantom model that simulates hardness similar to a human liver was used to test the US scan with the developed prototype. The result showed that the prototype could successfully perform the US scan with both pitch and yaw motions. The intrinsic elastic structure of the prototype was useful because it adequately fit the curved phantom surface without having to take extra care or to risk an

123

Int J CARS excess load on the phantom model. Rigidity of the manipulator was 1.69 N/mm at the initial position (pitch 20 degree and yaw 0 degree) when the load was applied against the US probe from the bottom. A vessel localization task was successfully performed in an in vivo test on a pig. The yaw motion was highly effective for investigating the vascular network because the manipulator allowed the US probe to turn while keeping the same position. This motion cannot be performed by a conventional laparoscopic US probe. Thus, the feasibility of the prototype is shown. Conclusion In this study, we proposed a laparoscopic US scan manipulator consisting of a newly developed spring-based elastic mechanism. The movement of two flat springs allows the prototype to perform 3 DOF motion at the tip (pitch, yaw and manual rotation along the long axis). The diameter of the prototype is 15 mm, and the US probe in the prototype is 12 mm in diameter. The intrinsic elastic structure allows the US probe to adequately fit the curved organ surface without extra effort of manipulation during the procedure. This avoids unexpected damage to the organs. The yaw motion was effective for investigating the vascular network, because the manipulator allows the probe to rotate while in the same position. This advantage would be useful in combination with a computer-aided navigation system. We are currently working on the development of a real-time navigation system that can perform 3D reconstruction of ultrasonographic images by implementing a magnetic position sensor (Aurora, Northern Digital Inc., Canada) at the tip of the manipulator. References [1] Konishi K, Nakamoto M, Kakeji Y, Tanoue K, Kawanaka H, Yamaguchi S, Ieiri S, Sato Y, Maehara Y, Tamura S, Hashizume M (2007) A real-time navigation system for laparoscopic surgery based on three-dimensional ultrasound using magnetooptic hybrid tracking configuration, Int J CARS 2:1–10. [2] Oguri S, Arata J, Ikeda T, Nakadate R, Onogi S, Akahoshi T, Harada K, Mitsuishi M, Hashizume M (2015) Multi-Degrees Of Freedom Laparoscopic Ultrasound Probe With Remote Center Of Mo- tion’’, Int J CARS, 10(Suppl 1):S242–244.

A miniature five finger hand robot for laparoscopic surgery: development of hand part and master glove R. Nakadate1, J. Arata2, S. Oguri3, S. Onogi1, T. Akahoshi4, T. Ikeda4, M. Hashizume1,4 1 Kyushu University, Center for Advanced Medical Innovation, Fukuoka, Japan 2 Kyushu University, Department of Mechanical Engineering, Fukuoka, Japan 3 Kyushu University, Innovation Center for Medical Redox Navigation, Fukuoka, Japan 4 Kyushu University, Center for Integration of Advanced Medicine, Life Science and Innovative Technology, Fukuoka, Japan Keywords Robotics  Master–slave  Laparoscopy  Sensor glove Purpose Laparoscopic surgery is now widely prevailing owing to its advantage for the patient quality of life. On the other hand, the conventional tools for the laparoscopic surgery are less intuitive to manipulate so that they requires more and more surgeon’s skill. In order to make them intuitive, the surgical robot has been

123

introduced and now widely accepted by the surgeons. Robots can provide more natural mapping between master and slave so that the operator can be free from considering transformation of the operating axis of tools. However, the shape of the end-effecter is still a conventional forceps, namely two nails. The surgical tasks such as suturing or knot tying are performed by special manner by using those tools, not as humans do by using fingers. Our motivation is to propose an ultimate version of the intuitive master slave surgical robot. Considering that the most proximal end of the master slave system is human hand, we suppose that the master slave robot which shape is the same as human hand is the most intuitive to control. In this study, we present a miniature five finger hand robot for laparoscopic surgery. Methods Firstly target specifications were defined. Target task was set as knot tying because it is one of the most complicated manipulations during surgery. As the application is laparoscopic surgery, the hand has to pass the trocar, which size is normally maximum 12 mm in diameter. We gave three actuated degrees of freedom (DOF) for each finger because it is minimum DOF for moving the finger tip at any position in three dimensional space. With one passive DOF, each finger has four DOF (15 actuated DOF plus five passive DOF in total in a hand) mimicking human fingers. Due to the size restriction, each finger could have only 2.3 mm in thickness. As the actuators have to be remotely placed, we employed wire transmission. In order to make the design of the links and joints simple, stacking concave and convex hinge was used. Proximal joint of each finger has two DOF by using arch convex in one side. One exception is the rolling joint of the thumb which is made of a pin hinge. Next, the master controller is an important part of this system. Although many sensor glove have been proposed and commercialized in the past [1], they are not cost effective. We have newly developed a 15 DOF sensor glove. It also employed the wire system for sensing the finger movement. For example, the distal end of the wire 1 is attached to the finger tip, go through the plastic tube fixed to the glove, and its proximal end is extended to the wrist. The wire 2 is attached to the proximal link of the finger. The wire 2 can measure the bend of the first joint. The wire 1 measure the total bend of first, second, third joint. The difference between wire 1 and 2 reflects the bend of the second, third joint. In the same way, swing motion of the finger is also detected. In this way total fifteen wires are put around the glove to detect corresponding joint to the robot hand compensating all interference between joints. Fifteen wire displacements at the wrist are measured by USB fish eye camera all at once. The merit of this method is cost effective and mechanically simple, while a small latency was observed in the camera capture and image processing. Results A miniature 15 DOF five finger hand robot has been successfully developed within the size restriction (less than 12 mm in cross section). Also 15 DOF sensor glove had good performance. Figure 1A shows a snapshot of the slave robot grasping a grain of rice and 12 mm trocar. Figure 1B shows the master glove. We have validated the performance of the master slave system by gesture and grasping test. Figure 2 shows that the slave robot was successfully performing the shleginger’s six grasping types, cylindrical, tip, hook, palmar, spherical and lateral grip (from A to G). The cost of the master glove was very low about JPY40 K which is mostly the camera cost. The latency of the master glove was 190 ms.

Int J CARS Conclusion In this study, we have developed a prototype of the miniature five finger robot hand and its master glove. The result was very promising. In the next step, we will implement wrist, elbow, shoulder, and build complete system with two hands, a 3D articulating camera controlled by a head mount display. Finally we are looking forward to achieve our mid-term goal, knot tying by hand. Further study includes the validation of the intuitiveness. The approach of a recent study of body ownership by using robotic tool [2] will be applied to this proposed robot. Acknowledgements This work was supported by Center for Clinical and Translational Research of Kyushu University, Japan. References [1] Laura D, Sabatini AM, Dario P (2009) A survey of glove-based systems and their applications. IEEE Trans on Syst Man Cybern C 38(4) 461–482. [2] Arata J, Hattori M, Ichikawa S et al. (2014) Robotically Enhanced Rubber Hand Illusion. IEEE Trans on Haptics 7(4): 526–532.

Development and validation for arthroscopic electrocautery

Fig. 1 A) Slave robot grasping a grain of rice with 12 mm trocar. B) Master glove

of

steerable

cannulas

K. Kim1,2, C. Park1, Y. Kim1, S. Kwon1,2, S. Kang1, D.- Y. Lee3, J. Kim3, K.- J. Cho3, I. Jeon4 1 Korea Institute of Science and Technology, Robotics and Media Institute, Seoul, South Korea 2 University of Science and Technology, Seoul, South Korea 3 Seoul National University, Seoul, South Korea 4 Asan Medical Center, Seoul, South Korea Keywords Steerable cannula  Minimally invasive surgery  Arthroscopic capsular release  Electrocautery Purpose We have developed three types of steerable cannulas for arthroscopic electrocautery to perform capsular release for the treatment of frozen shoulder. A typical electrocautery is straight and this shape makes it difficult to reach the target area for the surgery due to the curvature of the shoulder joint and humeral head. It also could be invasive to approach inferior glenoid with a straight surgical tool because of the presence of the axillary nerve (Fig. 1).

Fig. 2 Slave robot performing the schleginger’s six grasping types

Fig. 1 Arthroscopic approach in capsular release. A straight surgical tool cannot approach inferior glenoid

123

Int J CARS We developed three types of steerable cannula devices with a radius of curvature suitable for the arthroscopic shoulder surgery and capable of equipping with various surgical instruments inside (Fig. 2). A modified small electrocautery was equipped to each developed cannula, and then quantitative evaluation experiments and in vitro experiments were carried out.

Fig. 2 The steerable cannulas for arthroscopic electrocautery consisting of our three types of steerable cannula devices and modified electrocautery device Methods We developed three end-effectors that were steerable at the distal tip to get an access to inferior glenoid through anterior or posterior portals on the shoulder. We determined the workspace and specifications using the mean glenoid size of adults and the size of standard arthroscopic instruments. The specification was; outer diameter = 90 degrees, inner tool channel. The link type end-effector uses a finger-like mechanism which has ability to bend from 0 to 180 degrees in a narrow space. The endeffector is composed of four body parts and two link parts. The body parts are connected with three joints; each link part connects first and third, second and forth body part. The body parts and link parts make series of four-bar linkage, thus it moves like finger. The entire length and the diameter of this mechanism are 24 mm and 4 mm. It moves like human finger by wire pulling. The concentric tube type end-effector uses a simple mechanism with a patterned pre-curved nitinol tube and a straight tube. The inner-tube is a high-curvature nitinol (NiTi) tube, which gets engraved by laser machining so that its slits increases the bending flexibility of the tubes. The end-effector has 3 mm in outer diameter with a 1.6 mm tool channel. The wire driven type uses a Polyether ether ketone (PEEK) tube with several slits on its wall. Two wires are passing through the each side of the PEEK tube wall, and the PEEK tube is bent by pulling each wire. The end-effector has 2 degree of freedom and 4 mm outer diameter with a 1.5 mm tool channel. Each of three devices has small hand-held actuation unit with one or two motors and users could control the tip of the end-effector with one hand. We modified a commercialized electrocautery to equip to our three devices. Results We conducted evaluation experiments to investigate bending characteristics after equipping the modified electrode using shoulder model. The link type has a high curvature, but it has relatively complicated mechanism and it took higher cost and longer time to manufacture and assemble. Also, its bending characteristic was significantly decreased when equipped with the electrode due to the stiffness of it, so we concluded this mechanism was not suitable for steerable electrocautery.

123

The concentric tube type has a simple mechanism with bigger tool channel, thus it could be easily integrated with the electrode. The bending angle was slightly decreased when equipped with the electrode, but it was enough to approach to the target area. The wire driven type was relatively stiff and the bending characteristics did not change when equipped with the electrode. We conducted ex vivo experiments using a frozen porcine shoulder model which was anatomically similar to a human frozen shoulder. We compared maneuverability and cauterization characteristic of the conventional electrocautery and our steerable cannulas. The maneuverability was good enough when the cannulas were used for a short time, but as the operation time got longer than twenty minutes, the surgeon complained of fatigue on his wrist and shoulder. It was because the shape of actuation unit and the place of joystick button. We could successfully performed capsular release with the developed electrode and cannulas, but the whole operation took much more time than the conventional electrocautery device. It was because our electrode was smaller and it had less power (30 W) than the conventional one (60 W). Conclusion We have presented three types of steerable cannulas and its feasibility in arthroscopic capsular release. Using the two types of steerable cannulas, we were able to reach inferior glenoid and perform capsular release on frozen porcine shoulder model. The wire driven type showed better performance with stiffness and durability than the concentric tube type for shoulder application, but it has limitation with miniaturization. Thus, the concentric tube type could be a good candidate for using in micro joint such as wrist, finger and oral joint. Further studies are under way to develop improved prototypes that overcome some limitations from the experiment, including improving the design of actuation unit after hearing user’s opinion, and a new electrode with higher power.

Monitoring the progression of the fetus during labor M.-A. Vitrani123, J. Nizard1234, G. Morel123 1 Sorbonne Universite´s UPMC Univ. Paris 06, UMR 7222, ISIR, Paris, France 2 INSERM, U1150, Agathe-ISIR, F-75005, Paris, France 3 CNRS, UMR 7222, ISIR, F-75005, Paris, France 4 APHP Pitie´ Salpe´trie`re Hospital, Gynecology and Obstetrics Department, Paris, France Keywords Robotics  Gynaecology Purpose Monitoring of the progression of the fetus in the maternal pelvis is routinely assessed clinically. The growing literature on the clinical assessment during labor shows little reliability or reproducibility of clinical assessment [1, 2]. There is a need to develop tools that provide objective quantitative parameters of what is taking place in the maternal pelvis during labor. Among the tools that are being developed to help better clinical assessment, our work focused on a system called LaborPro (Trig Medical Ltd. Yokneam, Israel [3]). The studied system allows positioning of the fetal head in the maternal pelvis providing fetal head position and station. The aim of our research is to evaluate the reliability and the reproducibility of data provided by the LaborPro system on a model. Methods The LaborPro system uses magnetic position trackers and ultrasound technology controlled by PC-based software. A low power magnetic field is generated by a flat transmitter (microBIRD-SA, Ascension Technology, Inc, Burlington,VT, USA) that is placed under the mattress of the delivery bed (Fig. 1). Three miniature pose sensors sense the magnetic field created by the transmitter and the system controller calculates the position and orientation of each sensor in a

Int J CARS three-dimensional spatial configuration. A standard ultrasound system with a 2–5 MHz abdominal probe is connected to the computer, displaying real-time ultrasound images on its screen. A pose sensor is mounted on the ultrasound probe enabling the system, after a proper calibration, to identify the exact 3D location and orientation of each ultrasound pixel in space. In order to obtain the ultrasound images to determine fetal head position and fetal station some human manipulation is required.

Fig. 1 Schematic representation of the system For in vitro evaluation, we design a bench test with a fetal head and a maternal pelvis. A robot is used to position the fetal head with respect to the pelvis (Fig. 2). Station was measured for 12 different settings with 3 different trained specialists. For every setting, each operator measured the station as routinely done and registered the station provided by LaborPro System. Measurements were performed in several sessions over several days for each operator. For each set of measurements, we calculated the mean value, with standard deviation, and subsequently the coefficient of variation. Each parameter was used for intra- and inter-operator variability.

positioned by a robot programmed for predetermined fetal head positions and stations. This setting limits the possible variability that could be observed when several operators performed consecutive measurements, since fetal head could move during time, and according to uterine contraction. Moreover, operators performed the whole process of measurement for each data, since the robot positions the head back in the neutral position every time. This limits the theoretical artifact when all operators work on recorded images or volumes. The overall very small variability is encouraging for the development of non-invasive systems for station monitoring when all other parameters are controlled. Systems such as the one described here, with a robotic model for fetal head position and station, should help progress the development of non-invasive tools to monitor fetal head descent and station during labor. Conclusion The system described in this section, which combines ultrasound and a position-tracking system, was developed using bone morphing of the maternal pelvis. This bone morphing and the accuracy of distance measurements were validated on models, which do not take into account the accommodation phenomenon occurring during birth. Nevertheless, the in vitro validation of the intra- and inter-operator variability was good. References [1] Dupuis O, Silveira R, Zentner A, Dittmar A, Gaucherand P, Cucherat M, Redarce T, Rudigoz RC (2005) Birth simulator: reliability of transvaginal assessment of fetal head station as defined by the American College of Obstetricians and Gynecologists classification. Am J Obstet Gynecol 192:868–874. [2] ACOG Technical Bulletin No. 218, December 1995 (replaces no. 137, December 1989, and no. 157, July 1991). (1996) American College of Obstetricians and Gynecologists. Int J Gynaecol Obstet 53:73. [3] Trig Medical LtD http://www.trigmed.com

Ensuring the safety of minimally invasive image guided cochlear implantation T. Williamson1, N. Gerber1, C. Rathgeb1, W. Wimmer1, J. Anso1, M. Caversaccio2, S. Weber1, K. Gavaghan1 1 ARTORG Center for Biomedical Engineering, Bern, Switzerland 2 Bern University Hospital, ENT Head and Neck Surgery, Bern, Switzerland Keywords Robotics  ENT  Direct cochlear access  Safety

Fig. 2 Picture of the final system Results The in vitro data show that fetal head station determination using the automatic mode of the LaborPro has very little (less than 5 %) intraand inter-operator variability in a fixed robotic model. This information completes clinical data from the literature. This is the first time these new tools were tested on models where the fetal head is

Purpose Over the last decade numerous attempts at an image guided technique for gaining minimally invasive access to the inner ear have been made. Replacing a conventional mastoidectomy approach with a drilled tunnel only slightly larger in diameter than the implant electrode array (1.5–1.8 mm), minimally invasive cochlear implantation (MICI) poses numerous advantages, including the preservation of mastoid tissue and potentially reduced surgical and recovery times. A preliminary clinical study demonstrated the feasibility of the approach [1] however resulted in facial nerve damage in one of seven cases, thereby highlighting the need for an optimized and highly controlled drilling process. Additionally, with the critical need for high drilling accuracy, standard image guidance registration calculations and feedback prove to be insufficient. To this end, an image guidance robotic system with a multi-layer safety mechanism based on registration accuracy and external sensor information was designed to ensure drilling accuracy and patient safety. Integrated into an existing image guided for robotic minimally invasive cochlear implantation the safety mechanisms were implemented and verified in a study on cadaveric temporal bone samples.

123

Int J CARS Methods A drilling protocol designed to optimize the safety of the procedure was proposed and integrated into a custom planning software and robotic system [2]. Software allows the segmentation of the structures within the mastoid and the definition of the trajectory, with optimization of the drilling path completed based on the distances from the trajectory to surrounding critical anatomy. Once optimization is complete, the trajectory is automatically divided into multiple segments. The first segment commences at the mastoid surface and terminates 3 mm distally to the level of the facial nerve; the segment is assigned a drilling protocol (2 mm intervals) designed to provide sufficient thermal control whilst optimizing drilling time. During drilling of the first segment the forces observed at the tip of the drill are recorded and comparison of these forces with sampled surrounding bone density profiles allows the estimation of the tool pose independently from registration error [3]. Intraoperative cone beam CT imaging is performed prior to commencing the second segment which passes the close lying nerves. Based on the acquired images, the projected distance at which the current drill trajectory will pass anatomical structures, in addition to the drilling error, is calculated automatically using custom software. For distance calculations, a titanium rod inserted into the drill tunnel is automatically segmented and registered to the preoperative plan using a mutual information registration. The second segment terminates 3 mm past the level of the facial nerve and defines a region in which a heat sensitive drilling protocol, previously determined experimentally [4], is to be employed (0.5 mm drill intervals) to ensure thermal damage of the surrounding nerves is avoided The final segment completes the drilling to the middle ear cavity at the original drilling parameters as the drilling returns to a less critical region. The effectiveness of safety measures were assed in minimally invasive drillings performed on 16 temporal bone specimens. Registration of the specimen to the pre-operative images was required to meet heuristically determined error thresholds of FRE \ 0.04 mm and a maximum Euclidean difference from a leave-one-out registration test at the planned target \ 0.5 mm. Trajectories were drilled by the robotic system according to the planned protocols and safety measurements were performed on completion of the drilling of the first segment. Accuracy of the drilling, intraoperative image based safety calculations and force-density tool pose calculation as well as the preservation of anatomical structures were assessed on high resolution CT (xtremeCT, Scanco Medical, CH). Results Minimally invasive access to the middle ear, targeting the round window of the cochlea, was successfully performed in all 16 cases. Inability to assess one case on the postop CT lead to its exclusion from subsequent analysis. Registration accuracy thresholds resulted in the detection of a fiducial that had been mistakenly removed and returned to the same hole after preoperative imaging. The procedure was successfully performed after the acquisition of a second image. A drilling accuracy of 0.15 ± 0.07 mm was observed at the target on the round window and accuracies of 0.08 ± 0.04 mm and 0.12 ± 0.05 mm were observed on the surface of the mastoid and level of the facial nerve respectively. In all specimens the preservation of the facial nerve and chorda tympani was confirmed on postoperative high resolution CT images; in one case the chorda tympani was sacrificed due to a narrow facial recess. The drilling error predicted at the level of the facial nerve at the round window target point, based on the acquired intraoperative images were calculated with accuracies of 0.14 ± 0.1 mm and 0.19 ± 0.11 mm respectively. The distance from the drill to the facial nerve was estimated with an accuracy of 0.04 ± 0.04 mm, suggesting the major segmentation and prediction errors did not occur in the direction towards the nerve. The force-density algorithm predicted the actual tool path at 3 mm before the facial neve with an accuracy of 0.22 ± 0.14 mm (Fig. 1).

123

Fig. 1 Drilling safety mechanisms using (a) force and density data, and (b) image data (b), were assessed on post-operative microCT images (c) Conclusion Presented above is a drilling protocol and safety mechanisms designed to ensure the safe completion of minimally invasive cochlear implantation procedures. The described mechanisms are designed to prevent both mechanical and thermal damage to the structures of the facial recess and were evaluated on a total of 16 human temporal bone specimens; in all cases the structures of the facial recess remained intact according to the pre-operative plan. References [1] Labadie RF, Balachandran R, Noble JH, Blachon GS, Mitchell JE, Reda FA, Fitzpatrick JM (2014). Minimally invasive imageguided cochlear implantation surgery: First report of clinical implementation. Laryngoscope, 124(8), 1915–1922. [2] Bell B, Gerber N, Williamson T, Gavaghan K, Wimmer W, Caversaccio M, Weber S (2013). In Vitro Accuracy Evaluation of Image-Guided Robot System for Direct Cochlear Access. Otology & Neurotology, 34, 1284–1290. [3] Williamson TM, Bell BJ, Gerber N, Salas L, Zysset P, Caversaccio M, Weber S (2013). Estimation of tool pose based on force-density correlation during robotic drilling. IEEE TBME, 60(4), 969–76. [4] Feldmann A, Anso J, Bell B, Williamson T, Gavaghan K, Gerber N, Zysset P (2015). Temperature Prediction Model for Bone Drilling Based on Density Distribution and In Vivo Experiments for Minimally Invasive Robotic Cochlear Implantation. Annals of Biomedical Engineering. [Epub ahead of print]

Ex-vivo evaluation of a chorda tympani prediction model for minimally invasive cochlear implantation C. Rathgeb1, K. Gavaghan1, C. Du¨r2, O. Scheidegger3, T. Williamson1, L. Anschu¨tz2, M. Caversaccio2, S. Weber1, N. Gerber1 1 University of Bern, ARTORG Center for Biomedical Engineering Research, Bern, Switzerland 2 Inselspital Bern, Department of ENT, Head and Neck Surgery, Bern, Switzerland 3 Inselspital Bern, Department of Neurology, ENMG-Station, Bern, Switzerland Keywords Chorda tympani  Prediction model  Minimally invasive  Cochlear implantation Purpose Minimally invasive cochlear implantation (MICI) requires the definition of an access tunnel from the surface of the mastoid to the cochlea, passing through the facial recess. Bounded by the facial nerve, chorda tympani, the medial aspect of the external auditory canal and the incus, the definition of a safe drilling trajectory requires the accurate calculation of distances to these close lying structures to ensure sufficient safety margins are respected. Previously described planning systems calculate distances using automatically or semi automatically segmented anatomy from preoperative CT images.

Int J CARS However, to date, the inability to visualise the chorda tympani within the middle ear cavity on standard CT images has meant that the full structure cannot be assessed, possibly placing it at risk. To ensure a sufficient safety, a path prediction model of the chorda tympani based on the segmented section of the nerve passing through bone was developed and tested in a study on temporal bone specimens. Methods A petrous bone specimen was extracted from a human cadaver head preserved in Thiel and prepared for MRI scanning. The specimen was inserted into inert perfluoropolyether fluid (Solvay Solexis Fomblin Inert PFPE Fluid) and placed under vacuum for more than 12 h to remove possible air bubbles. This step ensured a good contrast between the chorda tympani and the middle ear cavity filled with Fomblin in MRI images. A high-resolution three-dimensional gradient echo 14T MRI scan of the petrous bone was acquired (GRE3D, isotropic resolution of 0.13 mm). The high resolution images allowed the visualisation of the chorda tympani not only in the bony structure but also in the middle ear cavity. The course of the chorda tympani inside the tympanic cavity and the transition to the tympanic orifice of canal for chorda tympani tympani were identified to form the basis of a prediction model based on cubic spline interpolation. The model was then used during planning of MICI. An previously developed planning software for robotic cochlear implantation was adapted to include the prediction model and the software was used to plan access tunnels for robotically performed MICI procedures on N = 23 temporal bones (17 preserved in Formalin and 8 preserved in Thiel). Planning of the procedure included segmentation of the chorda tympani in a two-step process. The posterior part of the chorda tympani encased in bone was segmented using previously described methods. The part inside the middle ear cavity was created using the above mentioned prediction model built using the selection of landmarks on the CT slices at the iter chordae posterius and the malleus processus anterior. The prediction model was used to assess the distance at which an access tunnel could be planned from the chorda tympani (see Fig. 1). The planned procedures were drilled utilizing a robotic system MICI, followed by cochlear electrode array insertion and postoperative assessment.

Fig. 1 Manually segmented facial nerve and chorda tympani on 14T MRI image (left). Plan of a drilling trajectory to the cochlea including the chorda tympani prediction model (right) Results The chorda tympani was successfully segmented in all specimens. In six cases, the chorda tympani was chosen to be sacrificed due to the small diameter of the facial recess in that sample identified during the planning of the drill tunnel. In all cases, postoperative endoscopic inspection was performed by a surgeon. In three of the six chorda tympani chosen to be sacrificed during planning, no damage was observed by the surgeon. The other three structures were reported to be damaged as expected by the plan. In two Formalin fixed samples, the chorda tympani was found to be damaged although the planned margin to the structures were suitable for drilling. Nevertheless, the surgeon reported very thin and fragile nerve structures due to the Formalin preservation liquid and

indicated that the chorda tympani might have been damaged during elevation of the tympanic membrane prior to endoscopic inspection. In postoperative visual inspection on micro CT images, the two structures were confirmed to have safe margins to the drilled tunnel which supports the hypothesis that the damage occurred after the drilling process. Conclusion Results of this pilot study suggest that the proposed prediction model is an effective method of determining the path of the chorda tympani through the middle ear cavity when it is not visible on preoperative images. The method seems sufficient to aid in the preservation of the structure during minimally invasive cochlear implantation and may be used for the planning of alternative otologic procedures in the future on further validation of its accuracy.

Thermal monitoring of the facial recess during drilling for minimally invasive cochlear implantation L. Fichera1, N. Dillon1, K. Kesler2, M. Zuniga Manrique3, J. Mitchell1, R. Labadie3 1 Department of Mechanical Engineering, 2 School of Medicine and 3 Department of Otolaryngology, Vanderbilt University, Nashville, United States Keywords Temporal bone drilling  Thermal monitoring  Cochlear implantation  Facial nerve Purpose Cochlear implantation traditionally involves a mastoidectomy to gain access to the cochlea for electrode insertion. Recently, a less invasive approach has been proposed, in which a narrow linear hole is drilled from the external skull surface to the cochlea. Pre-operative and intraoperative computed tomography (CT) scans are used to plan a safe path to the cochlea that avoids vital, bone-embedded structures such as the facial nerve. Based on this plan, the drill is aligned using a microstereotactic frame [1] or a robot [2]. Accurate image guidance and specialized hardware minimize the risk of contacting the vital anatomy with the drill; however, nerve injury can occur via thermal damage secondary to heat generated by the bit cutting through nearby bone and/or from friction between the drill bit and the surrounding bone or bushing sleeve. Prior work by others provided evidence that the temperature at the facial nerve can reach potentially harmful levels if additional safeguards (e.g. irrigation, planned trajectories through more porous bone) are not employed [3]. The purpose of the present work was to evaluate the heat generated by both manual and automated linear drilling near the facial nerve in human cadaveric temporal bones. Several sets of drilling parameters were employed with the intent that results would guide the selection of drilling parameters to be used clinically. Methods An experimental setup and procedure was developed to evaluate the temperature rise near the facial nerve while drilling a path to the cochlea. Cadaveric temporal bones were obtained from Science Care Inc. (Phoenix, AZ, United States), and 3–4 drilling experiments per bone were undertaken. For each experiment, the hardware and planning procedures similar to those described in [1] were used. A microstereotactic frame (‘‘Microtable’’) was manufactured and mounted to the specimen to align the drill along the desired path. Each specimen was cut along a plane perpendicular to the drill path in a region called the facial recess (Fig. 1a). The facial recess was selected because the drill passes very close to the facial nerve at this location. An infrared thermal camera (Flir A655sc, 50 lm close-up lens, Flir Inc. USA) was positioned to record the bone temperature at the cut plane (Fig. 1b) while drilling the medial stage of the path with

123

Int J CARS a 1.59 mm diameter stainless steel bit (CingleBit geometry with 86.9 point angle, Orchid Orthopedic Solutions, Inc.)

Fig. 2 Comparison between a manual and an automated drilling trial. The top plots show the position of the drill over time; the bottom plots report the mean temperature at the facial recess, at a distance of 0.5 mm from the surface of the drill bit. Temperature curves are surrounded by 95-percentile boundaries

Fig. 1 (a) CT scan of temporal bone specimen showing planned drill path (yellow), cochlea (purple), and facial recess plane (red) where temperature recordings were made; (b) experimental setup showing device hardware mounted to temporal bone and thermal camera position Drill spindle speed was set at 20,000 rpm. The drill advancement parameters were varied and performed both manually and using a motorized linear stage. Several pilot trials were performed at a constant advancement velocity and high temperatures were observed. This observation led to the hypothesis that a peck drilling strategy would be necessary to reduce heat generation. Six manual drilling trials were performed by an experienced surgeon using a peck drilling strategy consisting of advancing the drill approximately 1 mm per peck at approximately 0.5 mm/s with the drill retracted for approximately 3 s between pecks. Water irrigation was used during drilling. Subsequently, the automated drill press was programmed to mimic the manual peck drilling strategy and the temperature profiles were compared. Results Four manual and automatic drilling trials were performed. Fig. 2 shows an example for each case. The plots in the upper row show the position of the drill throughout the trial (the zero point represents the final position in the facial recess); the plots in the bottom row show the mean temperature observed at a distance of 0.5 mm from the drill surface, surrounded by 95th percentile (± 2r) boundaries. Data from a prior clinical study [1] was used to estimate 0.5 mm as the minimum distance between the drill bit surface and the facial nerve.

123

The surgeon reported using 0.5 mm/s as the reference speed, and adjusting it according to the force required, e.g. slowing down when the bone was harder. This strategy resulted in abrupt speed variations, such as the one observed in the second peck in the upper left plot of Fig. 2. Additionally, the manual control of the drill press used faster insertion and retraction speeds. Analysis of the temperature data revealed a pattern of progressive rises, synchronized with the movement of the drill press. The maximum temperature recorded during the manual trials was 64.92 C. The temperature peaks observed during the automated trials range from 58.84 C up to 128.42 C. Note that experiments were performed at room temperature (* 20 C); thus, all temperature data should be adjusted to account for starting at normal body temperature. Conclusion Results indicate that kinematic control of drill advancement (i.e. specifying velocities and peck parameters) alone is less effective for minimizing temperatures than hybrid force and position control as employed in the manual cases. This finding corresponds to the model developed in [3], which concluded that the thermal energy rate is proportional to force at the drill tip. Thus, automated drilling strategies should employ a hybrid force and position control scheme. The specific force thresholds and control law need further investigation. Our experimental results show that manual peck drilling results in a limited temperature elevation, thus indicating its potential to limit the risk of thermal injury to the facial nerve during minimally-invasive cochlear implant surgery. References [1] Labadie RF, Balachandran R, Noble JH, et al. Minimally invasive image guided cochlear implantation surgery: First report of clinical implementation. Laryngoscope. 2014 Aug;124(8):1915–22. [2] Bell B, Stieger C, Gerber N, et al. A self-developed and constructed robot for minimally invasive cochlear implantation. Acta Otolaryngol. 2012 Apr;132(4):355–60. [3] Feldmann A, Anso J, Bell B, et al. Temperature Prediction Model for Bone Drilling Based on Density Distribution and In Vivo Experiments for Minimally Invasive Robotic Cochlear Implantation. Ann Biomed Eng. Epub 2015 Sep 10.

Int J CARS Electrical impedance sensing to preserve the facial nerve during image-guided robotic cochlear implantation J. Anso´1, T. Williamson1, T. Wyss Balmer2, C. Precht3, H. Rohrbach3, N. Gerber1, M. Caversaccio4, S. Weber1, K. Gavaghan1 1 ARTORG Center for Biomedical Engineering, Bern, Switzerland 2 Institute for Surgical Technology & Biomechanics, Bern, Switzerland 3 VetSuisse Faculty, Bern, Switzerland 4 Inselspital Bern, Department of ENT Head and Neck Surgery, Bern, Switzerland Keywords Electrical impedance  Facial nerve  Robotic  Cochlear implantation Purpose In image-guided robotic cochlear implantation (CI) the surgeon relies on a highly accurate image-guidance system (accuracy \ 0.2 mm) to drill a *1.5 mm tunnel from the surface of the mastoid to the cochlea [1]. The drilling path may pass at distances as low as 0.2 mm to the facial nerve (FN), the most critical structure that needs to be preserved during cochlear implantation. In order to ensure the integrity of the FN, an image-guidance system for minimally invasive CI requires intraoperative safety measures independent from navigation. Currently a number of methods are proposed to enhance facial nerve safety during robotic CI including a redundant tool positioning algorithm based on correlations between drilling forces and bone density profiles [2], a neuromonitoring approach through the facial recess combining monopolar and bipolar measurements [3] and intra-operative imaging. The combination of these methods is expected to enable a more reliable measure of facial nerve safety, however additional or redundant safety methods may be useful to reduce uncertainty. Electrical impedance sensing has been proposed [4], [5] as a reliable approach to detect unsafe tool positioning during pedicle screw insertion. This technique is capable of detecting a transition between tissues with different electrical impedance (e.g. cancellous, cortical and soft tissue). Differences in electrical impedances among tissues is related to deviations in water contents amongst them. The facial nerve is surrounded by the fallopian canal, which has a much higher density than the nerve tissue itself. The ultimate goal of this study is to analyze feasibility of using electrical impedance to detect an unsafe trajectory during robotic CI. We hypothesize that changes in electrical impedance in the facial recess region correlate with variations in bone density profiles. If this hypothesis is positive, electrical impedance measurement could enhance the robustness and reliability of a safety network for robotic CI. Methods Electrical impedance is defined as the opposition of a volumetric medium to charge flow. It can be measured applying an alternate current (AC) through two electrodes covering the volume of interest and measuring the voltage drop between them. Resistance in a uniform conductor of length L and cross sectional area A is defined as R ¼ rho L=A ðohmÞ Resistance depends on the specific resistivity (rho) of the conductive material. The properties of bony tissue can be quantified from computer tomography (CT) measurements, whereby Hounsfield units (HU) are the standard units used for tissue density classifications in the human body. The mastoid presents variable densities in anisotropic distribution, mostly a combination of cancellous and cortical bone areas as well as the existence of air cells (1–2 mm spherical air filled cavities), and large variances among patients can be observed. To determine if changes in electrical impedance in the facial recess region correlate with variations in bone density profiles, electrical measurements were made using a multi-electrode probe during a live animal study in sheep (Bernese cantonal animal commission,

approval number 57/12). The measuring probe (1.8 mm Ø) had a working electrode (conical shape, r = 0.3 mm) at the tip, and three possible counter electrodes (rings) at distances d = 2, 4 and 7 mm from the tip. A needle electrode located in the ipsilateral mastoid surface enabled a fourth counter electrode configuration of the probe. A configurable system (MP150 and Stmisola, Biopac, USA) was used to measure electrical impedance by applying a current sinewave signal between each working and counter electrode configuration. In each of n = 5 animals, an image-guide CI robotic system and protocol [1] was applied to accurately drill up to 4 trajectories relative to the facial nerve. Per drilled tunnel, five measurement points were taken at controlled facial nerve distances, and at each point electrical impedance was measured at a frequency of 1 kHz and with an amplitude below 0.1 mA. Per trajectory and per electrode configuration, the cross-correlation between normalized impedance and normalized density values was evaluated. To determine the goodness of a potential fitting between impedance and density profiles, root mean square error (RMSE) was used. Results From 17 trajectories evaluated in five different subjects, a low correlation was found between impedance and density among all measured data points (Fig. 1). The average cross-correlation and RMSE scores through all 17 data sets was found to be 0.69 and 0.15 (Table 1). An example of impedance-density correlation pattern can be seen in Fig. 2.

Fig. 1 Correlation between resistivity (f = 1 kHz) and density values from CT scan for the electrode configuration of working to counter electrode distance d = 7 mm

Table 1 Correlation and fitting results per each of the four electrode configurations Counter electrode

Pearson’s corr. (r, p)

Crosscorrelation

RMSE

Concentric d = 2 mm

(0.47, 8.4e-6)

0.68 ± 0.11

0.15 ± 0.11

Concentric d = 4 mm

(0.52, 4.5e-7)

0.69 ± 0.11

0.15 ± 0.10

Concentric d = 7 mm

(0.54, 1.1e-7)

0.70 ± 0.09

0.14 ± 0.10

Far needle

(0.51, 8.5e-7)

0.69 ± 0.09

0.14 ± 0.10

Mean results

r = 0.51 ± 0.03, 0.69 ± 0.01 p \ 1e-5

0.15 ± 0.01

123

Int J CARS

Fig. 2 Example of a trajectory showing the similarity between the density and impedance profiles in the region of the fallopian canal (bony canal surrounding the facial nerve). In the top, an arbitrary CT slice along the trajectory is depicted Conclusion Electrical impedance could be a potential marker to enhance overall system’s safety during image-guided robotic cochlear implantation. An example of using this technique could be by combining it with drilling forces to further increase reliability of an existing pose estimation algorithm [2]. To further develop and verify this technique we propose to investigate integrated impedance sensing during continuous drilling into the mastoid. References [1] Bell B, Gerber N, Williamson T, et al. (2013) In Vitro Accuracy Evaluation of Image-Guided Robot System for Direct Cochlear Access. Otology & Neurotology 34: 1284–1290 [2] Williamson T, Bell B, Gerber N, et al. (2013) Estimation of tool pose based on force-density correlation during robotic drilling. IEEE Transactions Biomedical Engineering 60: 969–76 [3] Anso´ J, Du¨r C, Gavaghan K, et al. (2015) A Neuromonitoring Approach to Facial Nerve Preservation During Image-guided Robotic Cochlear Implantation. Otology & Neurotology (37): 89–98 [4] Dai Y, Xue Y, Zhang J, (2013) Drilling Electrode for Real-Time Measurement of Electrical Impedance in Bone Tissues. Annals Biomedical Engineering 42(3):1–10 [5] Bolger C, Carozzo C, Roger T, et al. (2006) A preliminary study of reliability of impedance measurement to detect iatrogenic initial pedicle perforation. Europe Spine Journal 15:316–320

Electrode array insertion for minimally invasive robotic cochlear implantation with a guide tube W. Wimmer1,2, K. Gavaghan3, T. Williamson3, N. Gerber1, M. Caversaccio1,2, S. Weber3 1 University of Bern, Artfifical Hearing Research, ARTORG Center, Bern, Switzerland 2 Inselspital Bern, Department of Otolaryngology, Head and Neck Surgery, Bern, Switzerland 3 University of Bern, Image-Guided Therapy, ARTORG Center, Bern, Switzerland Keywords Direct cochlear access  CI insertion models  Thiel vs. Formalin  Cochlear duct length

123

Purpose Minimally invasive robotic cochlear implantation demands an adapted surgical procedure when compared with conventional cochlear implant (CI) surgery. During insertion, the visibility and maneuverability of the electrode array is limited because of the small size of the direct cochlear access (DCA) tunnel (1.8 mm in diameter). It has previously been shown that a manual insertion of CI electrode arrays is feasible by introducing a tympanomeatal flap as an auxiliary access to the middle ear cavity [1]. To further reduce the invasiveness of the implantation procedure, i.e. to avoid the tympanomeatal flap, an insertion guide tube was developed to bypass the middle ear cavity and to enable a direct insertion of the array from the mastoid surface through the DCA tunnel. The aim of this ex vivo study was to evaluate the new implantation procedure for clinical applicability. Methods An insertion guide tube prototype that resembles the shape of the DCA drill was manufactured. It consists of 2 parts to enable removal after array insertion. Sixteen temporal bone specimens (8 Thiel fixed and 8 Formalin fixed) were prepared by placement of 4 fiducial landmark screws. Preoperative CT imaging was performed and drill trajectories were planned to align with the center of the round window. A high-accuracy robotic system was used to drill DCA tunnels (1.8 mm in diameter) in the temporal bones [2]. Free-fitting electrode arrays (28 mm length) were marked to achieve an angular insertion depth of 540 as calculated from the cochlear size of each specimen [3]. Manual insertion from the mastoid surface through the round window was performed through the insertion guide tube (Fig. 1). After insertion was complete, the guide tube was removed and the electrode lead was fixed. Postoperative cone beam CT and microCT imaging was performed to evaluate the insertion outcome.

Fig. 1 (Left) Formalin fixed temporal bone with insertion guide tube and inserted CI electrode array. (Right) Postoperative pseudocoronal cone beam CT slice showing the implantation outcome in a Formalin fixed temporal bone Results In all specimens the direct CI array insertions from the mastoid surface into the cochlea were feasible without complications using the guide tube. The removal of the insertion guide tube was possible without retracting the inserted arrays. One minor problem was the visualization of the insertion depth marks on the arrays at the level of the round window. All electrode arrays (n = 16) were inserted into the scala tympani, with an average angular insertion depth of 538 (Thiel fixed) and 409 (Formalin fixed). The arrays inserted into Thiel specimens showed a smooth array course, whereas in 4 Formalin specimens bending of the electrode array at the hook region occurred.

Int J CARS Conclusion The presented results show that CI array insertion through a removable guide tube seems to be feasible. The insertion guide tube serves as a base for further improvements of the implantation strategy. Further efforts must additionally consider a suitable method for sealing the electrode array after insertion and the management unforeseen events, such as bleeding or CSF leakage. The cochlear size based estimation of the insertion depths yielded promising results, however, must be evaluated in a clinical setting. Our results further indicate that Formalin fixed specimens are limited models for deep array insertions. References [1] Wimmer W, Bell B, Huth ME, Weisstanner C, Gerber N, Kompis M, Weber S, Caversaccio M (2014) Cone beam and micro-computed tomography validation of manual array insertion for minimally invasive cochlear implantation., Audiol. Neurootol., vol. 19, pp. 22–30 [2] Bell B, Gerber N, Williamson T, Gavaghan K, Wimmer W, Caversaccio M, Weber S, (2013) In Vitro Accuracy Evaluation of Image-Guided Robot System for Direct Cochlear Access., Otol. Neurotol., vol. 34, pp. 1284–1290 [3] Wimmer W, Bell B, Dhanasingh A, Jolly C, Weber S, Kompis M, Caversaccio M Cochlear duct length estimation: adaptation of Escude’s equation, 13th international conference on cochlear implants and other implantable auditory technologies, Munich, Germany

model of femur is taken as an input. To automatically locate an initial seed position, a rough shape of femur should be estimated. The femur center Of, orientation nf and femur length L are determined by its oriented bounding box. Then a plane passing through Of with normal vector nf is used to cut mid-diaphysis of the femur. The initial seed O is located at the mass center of the section contour to begin searching along n = nf. In each iterative step, the seed moves from the middle of the femur to the proximal end. The next optimal seed position is selected from a point set S in which every point satisfies a semi-spherical equation OP = r, pn \ 0, where r is the step length and p is the direction vector from O to P. The points can be ordered by their minimum distance R to the surface of the medullary canal. The point Popt with largest R becomes the next seed position O. If the seed reach end of the femur approximately OOf [ L/3 searching stop. If not, continue semi-spherical searching. At the end of iteration, a centerline and a list of corresponding radii are obtained to create a center tube.

Fully automatic determination of proximal femur morphological parameters and modeling of medullary canal Y. Wang1, L. Xu1, X. Chen1, Z. Taylor2, A. F. Frangi2, H. Zhang3, L. Wang3 1 Shanghai Jiao Tong University, School of Mechanical Engineering, Shanghai, China 2 Center for Computational Imaging and Simulation Technologies in Biomedicine, School of University of Sheffield, Sheffield, Great Britain 3 Shanghai Jiao Tong University, School of Medicine, Shanghai, China Keywords Femoral medullary canal  Semi-spherical searching  Proximal axis  Femoral isthmus Purpose Total hip arthroplasty and intramedullary nailing fixation have become the most successful surgical interventions among the orthopedic community. Accurate preoperative morphological measurements of the femoral intramedullary canal are an essential part of the preoperative plan and facilitate the selection or design of the most suitable femoral implant [1,2]. Traditional measurement methods of femoral morphology are primarily based on two-dimensional images, such as anterior-posterior X-ray films. Compared to 2D measurement methods, analyzing detailed osseous morphology of the femoral canal in 3D space should be a more accurate approach [3]. This study established an automatic method of measuring the morphological parameters and medullary canal modeling. Methods The processing and image rendering tools of the software are based on the open-source libraries Insight Toolkit and Visualization Toolkit. The proposed automatic computation is shown in Fig. 1. A surface

Fig. 1 The automatic computation pipeline and semi-spherical searching algorithm After the canal tube modeling, many key morphological parameters of the proximal femur can be easily accessed. The femoral isthmus locates at the tube section with minimum radius. From tube center points, a least-squares line is fitted to represent the femoral axis and a least-squares arc is fitted to represent the femoral arch. Results For the method evaluation experiments, the right femur was chosen. High-resolution computed tomography studies of 88 normal people (mean age of 43 ± 27 years, 51 male, and 37 female) receiving pelvic scans for reasons not related to orthopedic conditions were selected from our institution’s database. The average time cost for the entire algorithm procedure on standard computers was 1.085 s. To estimate the reliability of this method, 16 subjects (8 male and 8 female) were randomly selected from our main data set, and three trials were independently performed by three raters on all subjects. Average intra-class correlation coefficient scores were 0.9998, with a 95 % confidence interval of 0.9996 to 0.9999, indicating a robust performance (Fig. 2).

123

Int J CARS Accuracy of primary implant installation in fibula free flap maxillofacial reconstruction using the Alberta reconstruction technique H. Logan1,2, D. Aalto1,2, S. Nayar1,2, M. Osswald1,2, J. Harris1,2, D. O’Connell2, H. Seikaly2, J. Wolfaardt1,2 1 Institute for Reconstructive Sciences in Medicine, Edmonton, Canada 2 University of Alberta, Edmonton, Canada Keywords Surgical design and simulation  Fibula free flap  Implant  Maxillofacial reconstruction

Fig. 2 The canal tube (purple) and key measures in anterior-posterior (AP) view and left–right (LR) view. R is the radius of a maximal inscribed sphere at isthmus or the radius of posterior arch. k is a height ratio related to medullary canal of isthmus position or arch apex Conclusion In this study, a unique algorithm was developed to automatically analyze various parameters related to the proximal femur. A semispherical searching algorithm was used to determine a center tube in the femoral canal. Three-dimensional determination of femur morphology parameters provides more reliable reference in surgical application. A best-fit canal tube is also the modeling basis of patient specific femoral stem design. Acknowledgment This work was supported by the National Natural Science Foundation of China (Grant No.: 81201412, 81171429 and 81511130089), and Foundation of Science and Technology Commission of Shanghai Municipality (Grant No.: 14441901002, 15510722200). References [1] Tawada K, Iguchi H, Tanaka N, Watanabe N, Murakami S, Hasegawa S, et al. (2015). Is the canal flare index a reliable means of estimation of canal shape? measurement of proximal femoral geometry by use of 3d models of the femur. Journal of Orthopaedic Science Official Journal of the Japanese Orthopaedic Association, 20(3), 498–506. [2] Hawi N, Kenawey M, Panzica M, Stuebig T, Omar M, Krettek C, et al. (2015). Nail–medullary canal ratio affects mechanical axis deviation during femoral lengthening with an intramedullary distractor. Injury-international Journal of the Care of the Injured, 61(11), 2258–2262. [3] Su XY, Zhao Z, Zhao JX, Zhang LC, Long AH, Zhang LH, et al. (2015). Three-dimensional analysis of the curvature of the femoral canal in 426 chinese femurs. Biomed Research International, 2015(2), 1–8.

123

Purpose A great challenge remains with intuitive intraoperative microvascular jaw reconstruction. Errors in spatial positioning of the fibular osseous segments can often lead to unfavorable osseointegrated implant positions. The Alberta Reconstruction Technique (ART) was evolved from the Rohner Technique in order to undertake fully guided occlusion-based microvascular fibular free flap reconstruction of the jaws in patients with malignant disease. The ART procedure includes fully guided primary osseointegrated implant installation. Preoperative Surgical Design and Simulation (SDS) is carried out and patient specific fibular surgical cutting and drilling guides as well as a transfer template are designed using computer aided design software and manufactured using 3D printing technology (Fig. 1). The intention of the ART procedure is to reduce operative time, reconstruct the jaws with designed anatomical precision, reduce time to jaw reconstruction rehabilitation and improve functional outcomes. The purpose of the present study is to evaluate how accurate the ART procedure is by comparing preoperative planned osseointegrated implant positions to the achieved postoperative spatial positions.

Fig. 1 Alberta Reconstruction Technique toolkit. Top Left: Presurgical Model, Top Right: Planned Reconstruction Model, Center: Maxillary Cutting Guide, Bottom Left: Fibula Transfer Template, Bottom Center: Fibula drilling and cutting guide and Bottom Right: Reference Fibula Methods In the present study the post-operative medical CT images of 13 patients were digitally registered to the preoperative plan. 10 of the patients underwent maxillary reconstruction and 3 patients had mandibular reconstruction. Cartesian coordinates were obtained for each planned and actual implant position and the deviations between the corresponding coordinates were obtained. The deviations were transformed into the anatomical directions: the medial–lateral direction as pointing towards the mid-sagittal plane, horizontal direction

Int J CARS pointing from anterior to posterior (within axial CT plane), and vertical direction as pointing towards the occlusal plane (perpendicular to the CT axial plane). The deviation data was analysed along each anatomical direction by Student’s t-test to detect systematic errors in implant positioning. The confidence intervals were included in the analysis as well. Finally, the overall accuracy was described by calculating the distance and the angle between the planned and the actual implant positions. Results An average of 3.6 implants were installed per patient and a total of 47 implants were compared. Along the lateral-medial axis there was no systematic tendency of the actual implant position being more medial (mean = -0.5 mm, t = -1.7, df = 46, p = 0.09). The standard deviation of the differences between the actual and the planned positions was 1.9 mm. The largest deviation to the lateral was 3.8 mm and to the medial 4.4 mm. The actual implant positions in the horizontal direction were slightly more anterior than the planned implant positions (mean = 0.5 mm, t = -2.5, df = 46,p = 0.013, 95 % CI: 0.1 mm-1.0 mm) with a standard deviation of 1.4 mm. The largest deviation to the posterior was 3.7 mm and to the anterior was 4.4 mm. In the vertical direction there was a systematic difference between the planned and the actual vertical position of the implants. The implants were on average 2.1 mm closer to the occlusal surface than planned (mean = 2.1 mm, t = 7.7, df = 46, p Conclusion The ART produces predictable surgical and prosthetic rehabilitation outcomes for patients. The actual position of the implants corresponds well to the planned positions within the axial plane with no important systematic differences in the medio-lateral or antero-posterior directions. The results show that in the vertical spatial position, the implants were closer to the occlusal plane than planned. The reason for the systematic vertical discrepancy is not evident. Reduction in the prosthetic space could clinically affect the prosthetic treatment of the patient as it leaves limited space for superstructure design and ideal tooth arrangement. However, this will need to be assessed when these patients are seen for their prosthetic phase of treatment. Future research will be conducted in the clinical treatment of patients and how to reduce the deviation to increase the predictability of the outcomes. This information will shape future considerations in preoperative SDS for the ART.

Critical appraisal of patient-specific implants for orbital reconstruction R. Schreurs1, L. Dubois1, H. Essig2, E. Becking1, T. Maal1 1 AMC Amsterdam, Oral and Maxillofacial Surgery, Amsterdam, Netherlands 2 University Hospital Zurich, Oral and Maxillofacial Surgery, Zurich, Switzerland Keywords Orbital reconstruction  Trauma  Personalized medicine  Patient-specific implants Purpose The orbit is often affected in facial trauma. The goal of orbital reconstruction is to reconstruct the pretraumatic bony contour, restore orbital volume and to recover ocular function [1,2]. In more complex orbital fractures (Jacquie´ry Class III-IV) contour becomes important for adequate reconstruction. A patient-specific implant (PSI) may hold the best potential for optimal restoration. The complex 3D shape of the orbit can be accounted for in the design of a PSI by a complete digital workflow, allowing true-to-original anatomical repair [2,3,4]. PSIs also allow simplified insertion because of the perfect fit and increased stiffness [2,4,5] which decreases operating time [5], the incorporation of navigational points or rulers in the design for intraoperative navigation [2] and the possibility of overcorrection of soft-

tissue deficit [5]. In difficult late or secondary reconstructions, a PSI may obviate the need for additional osteotomies and bone grafts and may prevent multiple operative procedures [5]. An example of a design that utilizes the advantages of a patient-specific implant is visualized in Fig. 1.

Fig. 1 A PSI design with navigation points and rulers, for secondary orbital reconstruction (A). The implant size and rim extension ensure a compulsory fit. The rim extension accounts for the loss of zygomatic projection, preventing the necessity of an additional osteotomy (B) The predefined shape of the implant makes PSIs usable in computer-assisted surgery (CAS) and image-guided navigation. It is possible to acquire a perfectly shaped implant in the planning phase with a compulsory fit; the compulsory fit combined with intraoperative navigation should make perfect positioning of the implant achievable. Evaluation of implant placement can be performed either intraoperatively or postoperatively by image fusion of the preoperative planning and the Computed Tomography (CT) scan after implant positioning. Unfortunately, even with the use of all this technology small discrepancies are seen between planning and realization. A deviation between preoperative planning and result can originate from inaccuracies in the planning phase or positioning error in the intraoperative phase. The purpose of this study is to critically evaluate the postoperative result of a series of PSI reconstructions and assess the source of error leading to deviating implant placement, in order to better understand the effect of the inaccuracies arising in the workflow and to optimize future implant design. Methods A retrospective evaluation of all patients who underwent orbital reconstruction with a patient-specific implant at the Academic Medical Center Amsterdam, University of Amsterdam (AMC) from April 2014 to date was performed. A preoperative planning and subsequent design of a PSI had been made for all patients; in the planning software (iPlan v3.0.5, Brainlab AG, Feldkirchen, Germany) a stereolithographic model (stl) of the designed implant had been imported at the planned position. A CT scan was acquired postoperatively. The postoperative CT scan was imported in the planning software and fused with the preoperative CT scan. A segmentation of the implant in the fused postoperative scan was made and evaluation of acquired implant position was performed three-dimensionally based on the stl of the PSI and the segmented model. A critical evaluation of the possible errors in the workflow was performed if positioning of the PSI had not been performed precisely to plan. Inaccuracies were categorized in errors originating in the planning/ design phase and errors originating in the intraoperative phase. Results A total of 15 cases were evaluated. Possible explanations of deviating implant placement were found in the preoperative phase as well as the intraoperative phase. Inaccuracies in the planning phase led to an implant design which could not be positioned or fixated adequately

123

Int J CARS intraoperatively. A suboptimal implant design arose from a variety of sources, but could be further categorized in inaccuracies from CT scan acquisition, inaccuracies in the image processing phase or inaccuracies in the design phase. The effects on implant design and implant placement of scatter of osteosynthesis materials, mirroring inaccuracies, anatomical boundaries, screw hole location and extension size over the orbital rim are illustrated by clinical examples. In Fig. 2, the effect of implant design on implant positioning is visualized.

A surgical navigation system using an integrated image of preoperative and intraoperative MR images with sulci recognition for neurosurgery T. Noguchi1, I. Sato2, H. Shibata1, Y. Fujino2, H. Yamada3,4, T. Suzuki4,5, M. Tamura4, Y. Muragaki4, K. Masamune4 1 Future University Hakodate, Gradiate School of System Information Science, Hakodate, Japan 2 Future University Hakodate, Faculty of System Information Science, Hakodate, Japan 3 Murakumo Corporation, Tokyo, Japan 4 Tokyo Women’s Medical University, Institute of Advanced Biomedical Engineering and Science, Tokyo, Japan 5 Japan Association for the Advancement of Medical Equipment, Medical Device Strategy Institute, Tokyo, Japan Keywords Navigation  Brain tumor sergery  Sulci recognition  Functional brain mapping

Fig. 2 The effect of a rim extension on implant positioning. Planned (red) and acquired position (green) are shown. No extension (A) allows translation and rotation intraoperatively and can lead to a discrepancy between planning and result. If a rim extension is used (B), translation and rotations are limited, ensuring positioning according to plan Conclusion Although patient-specific implants hold great potential for precise true-to-original reconstruction of the bony orbit and correction of soft-tissue deficit, small deviations between planning and execution can be seen in the evaluation phase. Critical assessment of the possible sources of these inaccuracies helps to prevent or overcome design flaws and optimize the implant fit during surgery. The experience from the presented clinical cases can aid others in recognizing possible sources of inaccuracies and designing the perfect PSI. References [1] Dubois L, Steenen SA, Gooris PJJ, Mourits MP, Becking AG (2015) Controversies in orbital reconstruction—I. Defect-driven orbital reconstruction: A systematic review. International journal of oral and maxillofacial surgery, 44(3), 308–315. [2] Rana M, Chui CH, Wagner M, Zimmerer R, Rana M, Gellrich NC (2015) Increasing the Accuracy of Orbital Reconstruction With Selective Laser-Melted Patient-Specific Implants Combined With Intraoperative Navigation. Journal of Oral and Maxillofacial Surgery, 73(6), 1113–1118. [3] Gander T, Essig H, Metzler P, Lindhorst D, Dubois L, Ru¨cker M, Schumann P (2015) Patient specific implants (PSI) in reconstruction of orbital floor and wall fractures. Journal of Cranio-Maxillofacial Surgery, 43(1), 126–130. [4] Mommaerts MY, Bu¨ttner M, Vercruysse H, Wauters L, Beerens M (2015) Orbital Wall Reconstruction with Two-Piece Puzzle 3D Printed Implants: Technical Note. Craniomaxillofacial Trauma and Reconstruction. [5] Baumann A, Sinko K, Dorner G (2015) Late Reconstruction of the Orbit With Patient-Specific Implants Using Computer-Aided Planning and Navigation. Journal of Oral and Maxillofacial Surgery, 73(12), S101-S106.

123

Purpose In brain tumor surgery, functional brain mapping is done for maximum tumor resection and functional preservation. However, it is difficult to estimate the position to be stimulated by functional brain mapping. Therefore, it is effectiveness in displaying sulci, which are landmarks in functional brain mapping, to effectively support functional brain mapping and brain tumor resection. In our previous research, we had suggested a method to display the results of sulci recognition on a surgical navigation system. In our method, we directly recognize the types of sulci from an intraoperative magnetic resonance (MR) image and display positional relationship of a bipolar tweezers to sulci on the intraoperative MR image [1]. However, the method is unsuitable to use intraoperatively because it entails lengthy processing time. To solve this problem, we developed a new surgical navigation system that displays positional relationship of an electric probe for functional brain mapping to sulci. This system creates and displays an integrated image of preoperative and intraoperative MR images with sulci that recognized sulci types. In this paper, we describe this surgical navigation system and evaluate the effectiveness and processing time of the system. Methods We developed the surgical navigation system to display the positional relationship of the surgical instrument to the sulci, using an integrated image with sulci information for functional brain mapping (Fig. 1). This system consists of the computer (CPU: Intel Core i5-4670, RAM: 16 GB, OS: Ubuntu 14.04.1 LTS 64 bit, Windows 8.1 Pro) and a Brainlab navigation system (Curve, Brainlab AG). Preoperatively, this system creates a sulci image from a preoperative MR image. Intraoperatively, this system acquires the intraoperative MR image through a Vector Vision Link (V.V. Link), which is part of the Brainlab system. Then, a sulci image and a preoperative MR image, which are processed by non-rigid registration to intraoperative MR image, are integrated and displayed on the monitor. Then, surgical instrument position information is acquired in real time through the V.V. Link. Thus, it is possible to display the positional relationship of surgical instruments and sulci.

Int J CARS

Fig. 1 Overview of the developed system Processing by this system is achieved, roughly, in four steps: 1) Sulci recognition, 2) Pre-processing of non-rigid registration, 3) Nonrigid registration, and 4) Navigation. 1) Sulci recognition and 2) Preprocessing of non-rigid registration are processed preoperatively. 3) Non-rigid registration and 4) Navigation is processed intraoperatively (Fig. 2).

1) In sulci recognition, first, the preoperative image is de-noised using an unbiased non-local mean filter, and contrast of the image is corrected by linear transformation of gray-scale intensity levels. Then, sulci recognition is processed using BrainVISA [2]. In this way, we generate the three-dimensional volume image of sulci associated with each sulcus type with the colors of each sulcus. 2) In pre-processing of non-rigid registration, brain segmentation and creating the mesh is performed. Brain segmentation is processed roughly using BrainVISA and manually corrected using 3D Slicer. The mesh is created by Delaunay refinement process [3]. 3) In non-rigid registration, first, this system executes rigid registration of a preoperative image and an intraoperative image. Rigid registration uses an anterior commissure, a posterior commissure and tops of the lateral ventricles as feature points. Then, the preoperative MR image and the sulci image are transformed to the intraoperative MR image by non-rigid registration using Liu’s method [4]. Then, the intraoperative MR image and the sulci image, which are processed by non-rigid registration, are integrated. 4) In navigation, the integrated image is displayed on a monitor using a 3D Slicer. Then, positional information of the surgical instrument that has been acquired by using the Brainlab system is sent to the 3D Slicer using V.V. Link. Also, positional information of a bipolar tweezers is acquired while brain tumor are resected. In this way, the image-guide information is displayed on the monitor, combining the positional information of the probe or the bipolar, the sulci image, and the intraoperative MR image. Results We ran the system using log data (an intraoperative MR image, a preoperative MR image and positional information of a probe and a bipolar from Brainlab) of brain resection surgery outside the operating room. Then, we recorded the system behavior of 4) navigation displayed on the navigation monitor, and we showed it to a clinician. A clinician commented about the behavior of the navigation system as follows: ‘‘Displaying sulci on a MR image becomes landmark of a patient direction, because it is difficult to figure out a whole view of brain from a surgical microscope.’’, ‘‘If position and types of sulci are identified, result of preoperative planning enables to be used easily.’’ In addition, processing time of 3) non-rigid registration, which is processed intraoperatively, required 352 s. For purposes of clinical application, an intraoperative processing time of not more than 10 min is more practical. Therefore, processing time of the system is sufficient. Conclusion We developed a surgical navigation system that displays positional information of a probe to sulci on a preoperative MR image processed by non-rigid registration. We ran the system using log data of brain resection surgery to evaluate the effectiveness and processing time of this system. Results of evaluation experiment demonstrate the effectiveness of displaying sulci using this system, and the processing time of the system is sufficiency fast within 6 min. Therefore, we believe that our surgical navigation system could contribute to improvements in brain tumor resection while minimizing the loss of critical neurological functions. In the future, we aim for the realization of clinical application of the system after additional experiments. References [1] Noguchi T, Sato I, Fujino Y, Suzuki T, Tamura M, Muragaki Y, Masamune K ‘‘A Surgical Navigation System Guided by Intraoperative Cerebral Sulci Recognition,’’ International Journal of Computer Assisted Radiology and Surgery, Vol.10, Suppl.1, pp.S277-S278, 2015 [2] BrainVISA. http://brainvisa.info/. Accessed 2016/01/12 [3] Tournois J, Wormser C, Alliez P, Desbrun M (2009) ‘‘Interleaving Delaunay refinement and optimization for practical isotropic tetrahedron mesh generation’’, ACM Transactions on

Fig. 2 Workflow of the developed system

123

Int J CARS

[4]

Graphics, 2009; 28(3):75:1–75:9, SIGGRAPH ‘2009 Conference Proceedings Liu Y, Kot A, Drakopoulos F, Yao C, Fedorov A, Enquobahrie A, Clatz O, Chrisochoides NP ‘‘An ITK implementation of a physics-based non-rigid registration method for brain deformation in image-guided neurosurgery’’, Frontiers in Neuroinformatics, 2014; 8(33):1–10.

Real-time ultrasound Doppler enhances precision in image-guided approaches to the cerebello-pontine angle S. K. Rosahl1 1 HELIOS Klinikum, Department of Neurosurgery, Erfurt, Germany Keywords Craniotomy  Ultrasound Doppler  Image-guidance  Retrosigmoid approach

measured distance between the location of the sinuses as indicated given by the navigation system and the real location as indicated by the Doppler probe. The depth of the ultrasound probe was set at 2 mm, resulting in spatial accuracy of 1 mm when the mean thickness of the dura mater was subtracted. The shape and the cross-section area of the sinus were determined by ultrasound measurements. Visual detectability of the sinuses under the operating microscope was also noted. Finally, the time required to prepare and apply image guidance and ultrasound Doppler was measured for this specific procedure. Results Accidental incision of the transverse or sigmoid sinuses did not occur in any case when the two localizing methods have been used in combination. Image guidance was off-target by a mean of 2.64 mm (range 0–6 mm, SD 1.55 mm, D in Fig. 1). By the help of the micro-Doppler, the extent of craniotomies could always be limited to the most medial und inferior margins of the respective sinuses (Fig. 2).

Purpose The edges of the transverse and the sigmoid sinus mark the border for the most common approach to the posterior cranial fossa and the cerebello-pontine angle, the retrosigmoid approach. Injury to these anatomic structures bears a potentially lethal risk. The asterion, i.e. the junction of the lambdoid, temporooccipital, and occipitomastoid bone sutures is not a safe outer landmark for the transition of the transverse to the sigmoid sinus inside the skull since this junction may vary by as far as 10 mm (Fig. 1). We have previously shown that image guidance based on pre-operative magnetic resonance (MR) and computed tomography (CT) scans is superior to other methods in locating these landmarks before the craniotomy [1, 2, 3, 4]. In addition to the fact, that the dural coverings of the sinuses may obscure these venous structures completely, the margins of error of current image guidance systems in daily routine demand for real-time guiding methods in the operating room to enhance precision. We have investigated the applicability of a micro-Doppler ultrasound probe for this purpose. Fig. 2 A MRI showing a hypoplastic transverse sinus on the left side. B Projection of the landmarks to the skin based on image-guidance. C Micro-Doppler probe attached to a navigation probe in search for the sinus after craniotomy. D Doppler signal directed towards the probe

Fig. 1 Schematic of a retrosigmoid craniotomy along the edges of the transverse and sigmoid sinus on the right side D…distance between the real border of the sinus as confirmed by Doppler ultrasound and the virtual border as delineated by image guidance Methods In a series of 25 patients undergoing a surgery for lesions in the cerebello-pontine angle we have dispatched both image guidance and a micro-Doppler ultrasound probe (16 MHz, Multi-Dop pro, Compumedics, Germany) to the task of locating the transverse and sigmoid sinus. The mean maximum error of the image guidance system (Kolibri, Brainlab, Germany) was calculated from the

123

The exact location of the transverse sinus was invisible in 7 cases, while the sigmoid sinus was visually undetectable in one case. The shape of the cross section of the sinuses resembled a flat triangle in 23 of the cases. The ultrasound Doppler indicated blood flow outside the visible borders of the sinuses in 5 cases. Conclusion A combination of MR- or CT-based image-guidance and intraoperative micro-Doppler allows for precise location of the transverse and sigmoid sinus in retrosigmoid approaches to the posterior cranial fossa. The method prevents injury to these important venous structures by restricting the craniotomy strictly to the very edges of the sinuses and by indicating blood flow outside the borders of the sinuses as detected by preoperative imaging methods before the dural incision is performed. Both radiological imagebased and ultrasound Doppler guidance do not add operation time to the approach when used routinely. The method can be applied to all craniotomies that border on a venous sinus. Companies could easily facilitate the procedure by manufacturing an attachable guide for the micro Doppler probe to the standard navigation probes. References [1] Gharabaghi A, Rosahl S K, Feigl G C, Liebig T, Mirzayan J M, Heckl S, Shahidi R, Tatagiba M, Samii M. Image-guided lateral

Int J CARS

[2]

[3]

[4]

suboccipital approach: part 1-individualized landmarks for surgical planning. Neurosurgery 2008a; 62(3 Suppl 1): 18–22. Gharabaghi A, Rosahl S K, Feigl G C, Safavi-Abbasi S, Mirzayan J M, Heckl S, Shahidi R, Tatagiba M, Samii M. Image-guided lateral suboccipital approach: part 2-impact on complication rates and operation times. Neurosurgery 2008b; 62(3 Suppl 1): 24–29. Gharabaghi A, Rosahl S K, Feigl G C, Samii A, Liebig T, Heckl S, Mirzayan J M, Safavi-Abbasi S, Koerbel A, Lowenheim H, Nagele T, Shahidi R, Samii M, Tatagiba M. Surgical planning for retrosigmoid craniotomies improved by 3D computed tomography venography. Eur J Surg Oncol 2008c; 34: 227–231. Rosahl S K, Gharabaghi A, Hubbe U, Shahidi R, Samii M. Virtual reality augmentation in skull base surgery. Skull Base 2006; 16(2): 59–66.

approved the research and each patient was informed of the study and signed consent prior to surgery. In this abstract, we describe one of the most recent cases where we found the surgeon to be much more comfortable using augmented reality, more familiar with the visualization and therefore, more inclined to use the visualization at a number of different points in the surgery. The particular case was a craniotomy for a 77-year-old female patient undergoing resection of a right frontal meningioma. Prior to surgery a virtual model of the tumour was created from a manual segmentation of the T1 weighted gadolinium enhanced MRI using ITK Snap (Fig. 1). In the operating room AR was used on the skin/hair, bone, dura and on the cortex. AR visualizations involve acquiring a live sequence of camera images that are combined with the virtual tumour model. The transparency of the camera image is set such that the virtual tumour is visible below the real surface of thee patient. Furthermore, edges are extracted from the image, using a Sobel filter, in order to aid depth perception and to make the tumour appear below the surface rather than floating above it [4].

Intraoperative craniotomy planning for brain tumour surgery using augmented reality M. Kersten-Oertel1, I. J. Gerard 1,2, S. Drouin 1,2, J. A. Hall 3, D. L. Collins1,3,2 1 Montreal Neurological Institute, McConnell Brain Imaging Centre, Montreal, Canada 2 McGill University, Biomedical Engineering, Montreal, Canada 3 Montreal Neurological Institute, Neurology and Neurosurgery, Montreal, Canada Keywords Augmented reality  Image-guided neurosurgery  Craniotomy  Brain tumour Purpose Augmented reality (AR) visualization in image-guided neurosurgery (IGNS) allows a surgeon to see rendered preoperative medical datasets (e.g. MRI/CT) from a navigation system merged with the surgical field of view. Combining the real surgical scene with the virtual anatomical models into a comprehensive visualization has the potential of reducing the cognitive burden of the surgeon by removing the need to map preoperative images and surgical plans from the navigation system to the patient. Furthermore, it allows the surgeon to see beyond the visible surface of the patient, directly at the anatomy of interest, which may not be readily visible. In recent work we explored the usefulness of AR visualizations for specific tasks in the context of neurovascular surgery for arteriovenous malformations and aneurisms [1]. We found that AR was particularly useful for tailoring the craniotomy, planning a dissection path to the anatomy of interest, and intra-operatively differentiating between veins and arteries. In this work we explore the use of augmented reality visualization for craniotomy planning in brain tumour surgery. Craniotomy planning involves three stages, determining (1) the size and site of the skin incision, (2) the size and shape of the bone to be removed and (3) opening the dura to expose the brain. Methods Our AR IGNS system [2] is comprised of three components: (1) a custom-developed neuronavigation system, (2) a Sony HDR-XR150 video camera, which is equipped with an infrared tracker, and used for obtaining live images of the surgical scene, and (3) an optical tracking system. A detailed description of the system including the calibration, registration, and rendering of the AR view and its use in the operating room in neurovascular cases is described in [2, 3]. The system was used in 6 surgical cases for tumour resections by two different neurosurgeons. The MNI Research and Ethics Board

Fig. 1 Pre-operative images of the patient from our AR IGNS system Results For the AR IGNS system the camera calibration mean reprojection error was 0.90 mm (SD 0.47 mm) and the patient-to-image RMS registration error was 2.75 mm suggesting an accurate overlay between the real camera images and the virtual anatomy. The surgeon verified the alignment using manual inspection and the commercial Medtronic StealthStation S7 (Medtronic, Louisville, CO, USA). The patient-to-image registration error on the Medtronic system was 3.2 mm. The AR visualization was used by the surgeon to see the extent of the tumour below the visible surface of the patient. Specifically, AR was used on the skin, bone, and dura to facilitate the planning the shape and size of the craniotomy. On the skin (Fig. 2A), the boundaries of the tumour are drawn and the size of the craniotiomy is planned to go beyond the extent of the tumour margin. On the bone (Fig. 2B), the surgeon used the AR visualization to outline the tumour in order to plan the size of the bone flap to be removed and the locations of the burr holes to be made. The surgeon also used AR on the dura (Fig. 2C) prior to opening the dura and lastly on the cortex prior to dissection and resection of the tumour (Fig. 2D).

123

Int J CARS [4]

Kersten-Oertel M, Chen SS, Drouin S, Sinclair DS, Collins DL, ‘‘Augmented reality visualization for guidance in neurovascular surgery,’’ Stud Health Technol Inform, vol. 173, pp. 225–9, 2012.

Digitized brain-mapping localization and their compiled brainfunction database can lead the future-predicting glioma surgery M. Tamura1,2, T. Maruyama1,2, J.- F. Mangin3, I. Sato4, M. Nitta1,2, K. Yoshimitsu1, Y. Konishi1, J. Okamoto1, S. Ikuta1, K. Masamune1, H. Yamada1, S. Minami1, T. Kawamata2, H. Iseki1, Y. Muragaki1,2 1 Tokyo Women’s Medical University, Faculty of Advanced TechnoSurgery (FATS), Institute of Advanced Biomedical Engineering and Science (TWIns), Tokyo, Japan 2 Tokyo Women’s Medical University, Department of Neurosurgery, Neurological Institute, Tokyo, Japan 3 Neurospin, Biomedical Imaging Institute, CEA, The Computer Assisted Neuroimaging Labratory, Gif/Yvette, France 4 Future University Hakodate, Faculty of System Information Science Engineering, Hakodate, Japan Fig. 2 The surgeon used AR for craniotomy planning on the skin (A), bone (B) & dura (C). A: The orange arrow indicates the posterior boundary of the tumour and the blue indicates the planned posterior boundary of the craniotomy. The yellow arrow shows the medial extent of the tumour, which is also the planned craniotomy margin. B: the surgeon uses the augmented reality view to trace around the tumour in order to determine the size of the bone flap to be removed. C: AR is used prior to Conclusion We successfully brought our AR IGNS system into the OR for a number of tumour resection cases and explored the use of AR for the surgical task of craniotomy planning. Based on comments from the surgeon this type of visualization is useful for tailoring the size and shape of the craniotomy. By using AR visualization on the skin, bone and dura, the surgeon can see the extent and margins of the tumour and its location relative to his/her view of the patient. The visualization of the anatomy of interest below the visible surface of the patient facilitates determining the most appropriately sized craniotomy for each surgical case. In future work we will explore the use of AR for other surgical tasks involved in tumour resections such as planning dissection corridors and ensuring gross total resection. References [1] Kersten-Oertel M, Gerard IJ, Drouin S, Mok K, Sirhan D, Sinclair DS, Collins DL, ‘‘Augmented Reality for Specific Neurovascular Tasks,’’ in Augmented Environments in Computer Assisted Interventions, Munich 2015. [2] Kersten-Oertel M, Gerard IJ, Drouin S, Mok K, Sirhan D, Sinclair DS, Collins D, ‘‘Augmented reality in neurovascular surgery: feasibility and first uses in the operating room,’’ Int J Comput Assist Radiol Surg, Feb 26 2015. [3] Kersten-Oertel M, Chen SS, Drouin S, Sinclair DS, Collins DL, ‘‘Augmented reality visualization for guidance in neurovascular surgery,’’ Studies in health technology and informatics, vol. 173, pp. 225–9, 2012.

123

Keywords Brain mapping  Navigation  Digitization  Glioma surgery Purpose The purpose of this study is to manage the digitized intra-operative imaging and their compiled brain-function database for the futurepredicting glioma surgery based on patient’s future perspective depending on the tumor resection rate as well as the post-operative complication rate. From 2000, we operated more than 1500 patients of mainly glioma in ‘advanced information-guided surgery’ with application of the intraoperative MRI, updated neuronavigation, and histological and functional monitoring [1]. Additionally, from 2004, a dedicated device called IEMASTM (Intraoperative Examination Monitoring system for Awake Surgery) was developed and incorporated into more than 300 clinical brain-mappings by electric stimulation during awake craniotomy for eloquent lesions without interruption of the surgical manipulations [2]. This visualization of intraoperative information takes the most important part in the process to acquire objective evaluation and to integrate visual data as digitized data for intraoperative surgical decision-making. The previous brain-mapping localization in the electrical stimulator was not digitized data but simple visualized signal, therefore, it was difficult to reflect intraoperative MRI. To digitize the electric stimulation, the probe localization is recorded as a log file with brain-mapping analysis for informative database in neurosurgical field widely. Methods In Awake craniotomy, 1) successful acquisition of log data with the location of medical device integrated into intra-operative MR image was performed in ten cases and 2) evaluation of language-related location responding to surgical operator’s electrical stimulation and examiner’s task from the precise process analysis in medical device IEMASTM was performed (Fig. 1).

Int J CARS

Fig. 1 Awake craniotomy for brain mapping ingliomasurgery 1) In the navigation system (Brain Lab Corporation) integrated between updated intraoperative MRI and the probe location of electrical stimulation (Ojemann: Integral Corporation), the localization log files were acquired and visualized on the intraoperative MRI as digitized information using the image analyzing software (3D-slicer) in ten clinical cases from May 2015 (Fig. 2).

IEMASTM. In addition, intraoperative digitized location could transfer to per-operative patients and normalized brain surface. 2) The precise speech arrest (SA) area was confirmed in 54 cases (of 85) in the intraoperative brain mapping with electrical stimulation. Thirty four patients (63 %) were located near Broca area (threshold 4.6 ± 1.3 mA, at 31 N and 11 V task). Negative motor area was confirmed in 16 (30 %) patients at 5.1 ± 1.3 mA threshold, and in seven (13 %) cases SA was not confirmed on the cortex. SA in anterior language area was detected in 47 (87 %) and negative motor and Broca area were discriminated in 40 (74 %). This rate was affected by the intraoperative language task and tumor location. Conclusion Brain mapping accuracy should be always evaluated to clarify brain function in glioma patients within the limited stimulation time. Intraoperative digitized localization datasets for cortical and subcortical brain mapping are acquired precisely in glioma neurosurgery. Integrated image fusion and transformation, normalized datasets compiled brain function can lead the future-predicting glioma surgery in which to show the individual brain function and the complication risk for maximum glioma resection. References [1] Muragaki Y, Chernov M, Yoshimitsu K, Suzuki T, Iseki H, Maruyama T, Tamura M, Ikuta S, Nitta M, Watanabe A, Saito T, Okamoto J, Niki T, Hayashi M, Takakura K: InformationGuided Surgery of Intracranial Gliomas: Overview of an Advanced Intraoperative Technology. Journal of Healthcare Engineering in Medicine 3: 551–569, 2012 [2] Tamura M, Muragaki Y, Saito T, Maruyama T, Nitta M, Tsuzuki S, Iseki H, Okada Y: Strategy of Surgical Resection for Glioma Based on Intraoperative Functional Mapping and Monitoring. Neurol Med Chir (Tokyo) 55: 383–398, 2015

A framework for the identification and classification of depth electrodes R. Zelmann1, B. Frauscher2, J. Gotman2, D. L. Collins1 1 Montreal Neurological Institute, McConnell Brain Imaging Centre, Montreal, Canada 2 Montreal Neurological Institute, EEG, Montreal, Canada Keywords Depth electrodes  Electrode modelling  Tissue classification  Segmentation

Fig. 2 The Acquisition of log data with the location of the probe added to neuronavigation system 2) One hundred and seventy glioma patients from 171 cases in awake craniotomy performed between Jan 2010 and November 2015 were operated (106 males, 64 females in sex and 39.1 years old in average). In these cases, 85 left-hemisphere initially operated gliomas were analyzed. As a linguistic evaluation task, picture naming (N) and verb generation (V) task were mainly used in synchronizing with Ojemann electrical stimulator (frequency: 50 Hz, duration: 0.5 ms, bi-phasic 2–8 mA) to confirm speech arrest (SA) and to discriminate negative/positive motor area from SA if possible. Results 1) The digitized information located on the electrical stimulation using image analysis software (3D-slicer) could be achieved in all 10 cases (5 males, 5 females in sex and 40.0 years old in average). In some cases, accurate probe location could be confirmed for the inaccurate data in our past evaluation only using video analysis with

Purpose Epilepsy surgery is the therapy of choice for patients with therapyrefractory epilepsy and a suspected focal generator in non-eloquent cortex. The goal of this surgery is the removal of the epileptic focus to render the patient free of seizures. As part of the pre-surgical evaluation, intracranial depth-electrode electroencephalography (iEEG) is required in some of these patients. Depth electrodes are implanted into the brain through holes in the skull to accurately locate the focus. Each electrode consists of 8–15 contacts to record iEEG from the suspected target and grey matter (GM) along the trajectory. IEEG recorded from these contacts is considered the gold standard for focus localization. However, the anatomical location of each contact is only roughly assessed based on the visual analysis of post-implantation (i.e. with the electrodes) CT or MR images. Precisely locating the contacts is important as contacts deep in white matter (WM) or within cerebrospinal fluid (CSF) would not record iEEG. In this study, we propose an interactive tool to semi-automatically locate the depth electrodes, and to automatically identify the anatomical location (WM, GM or CSF) of each contact. We integrated our system to an image guided neurosurgery platform [1,2] providing the surgeons with this important information at the time of the surgery.

123

Int J CARS Methods We randomly selected ten electrodes (one electrode per patient), with a total of 148 contacts. An MRI before the implantation (preMRI) and an MRI with the implanted electrodes (posMRI) were used. Based on the target point (i.e. the electrode tip) and a second point along the electrode, the system modeled the physical characteristics of the electrode, including the volume occupied by each contact. GM, WM and CSF on each patient were automatically segmented [3]. The preMRI data was registered to a standard stereotaxic model [4] and the electrode locations from the posMRI were transformed to this stereotaxic space. Each contact was classified as GM, WM, CSF or outside the brain. We evaluated the two steps of our system in the following way. First, we compared the manually marked contacts to the proposed electrode model. A board-certified neurologist visually marked each contact of each electrode on the posMRI. For the semi automatic approach, it was sufficient to identify only two points: the target and another point of the electrode. As the contacts should form a straight line, we evaluated (sign test) the co-linearity and spacing of the manually identified contact 3D coordinates. Statistical threshold was set at 0.05. Second, to evaluate the classification of the anatomical location of the contacts, we compared the automatic method to the classification performed by an expert. Since it was important to classify the same contacts (i.e. same coordinates) by both methods, we considered as gold standard the neurologist’s visual classification of the contacts obtained from the electrode model (provided by our system in step 1). In this way, it was possible to focus the evaluation solely on the performance of the tissue classification step. Results Manually marked contact locations differ from a straight line as shown in Fig. 1A. The Euclidean distance between consecutive contacts was statistically different to a median of 3.5 mm (the real distance as specified by the manufacturer) for four electrodes (RLi:p = 0.004; LOF1:p = 0.0002; LHP p = 0.004; LOF2:p = 0.002). This suggests that the modeling step is particularly useful for oblique electrodes, probably because the contacts are not usually visible within a single plane or individual contacts were not visible in the reconstruction. Note in Fig. 1A similarity to the model for RH and difference for LOF and RLi. Figure 2B shows examples of obtained electrode models transformed to stereotaxic space.

Fig. 1 A) Manually marked contacts and semi-automatic model. Axes represent XY coordinates in original space. Visual markings shown as colored circles; electrode models with center of contacts in black. B) Electrode models from our system, after transformation to stereotaxic space on brain template

123

Fig. 2 A) Agreement of contacts’ classification between automatic and manual methods. B) Model of electrode RA obtained by our system rendered with segmented amygdala (orange) and hippocampus (green). Other segmented structures were removed from the view for clarity of display There was agreement between manual and automatic methods for the contact’s classification in 126 out of the 148 contacts (85 %; range 64–100 %; Fig. 2A). Sixty eight contacts out of 148 were automatically identified as WM (46 %) and 40 as GM (27 %). Similarly, 60/148 contacts were identified as WM (41 %) and 45 as GM (30 %) with the manual method. Discrepancies seem to be mainly for contacts laying at the boundary between GM and WM. Figure 2B shows an example of an electrode aiming at the amygdala and some automatically segmented GM regions. Note that the deepest 3 contacts are located in GM (amygdala). Conclusion Our system enabled modeling of implanted depth electrodes and the automatic location of each contact on segmented anatomical structures. Based on minimal expert intervention an accurate model estimate is obtained. Our system seems particularly useful for modeling oblique electrodes. Classification accuracy was very good. Next steps include validation in a larger cohort and classification with respect to segmented sub lobar regions. Our system could help improving the interpretation of iEEG, leading to better seizure outcome. References [1] Mercier L, Del Maestro RF, Petrecca K, Kochanowska A, Drouin S, Yan CX, Collins DL (2011) New prototype neuronavigation system based on preoperative imaging and intraoperative freehand ultrasound: system description and validation. Int J Comput Assist Radiol Surg 6(4):507–522. doi: 10.1007/s11548-010-0535-3 [2] Zelmann R, Beriault S, Marinho MM, Mok K, Hall JA, Guizard N, Haegelen C, Olivier A, Pike GB, Collins DL (2015). Improving recorded volume in mesial temporal lobe by optimizing stereotactic intracranial electrode implantation planning. Int J Comput Assist Radiol Surg. 1599–615. doi: 10.1007/s11548-015-1165-6. [3] Collins D, Zijdenbos A, Baare´ W, Evans A (1999) ANIMAL + INSECT: improved cortical structure segmentation. In: Information processing in medical imaging. Springer, pp 210–223 [4] Mazziotta J, Toga A, Evans A, Fox P, Lancaster J, Zilles K, et al. (2001). A probabilistic atlas and reference system for the human brain: International Consortium for Brain Mapping (ICBM). Philosophical transactions of the Royal Society of London Series B, Biological sciences 356 (1412):1293–1322. doi:10.1098/rstb.2001.0915

Int J CARS An analysis of the effect of 3D printed renal cancer models on surgical planning T. Rude1, N. Wake1, D. K. Sodickson1, M. Stifelman1, J. Borin1, H. Chandarana1, W. C. Huang1 1 New York University Langone Medical Center, Urology, New York, United States Keywords 3D printing  Surgical modeling  Urology  Robot assisted surgery Purpose Pre-operative three-dimensional (3D) printed renal malignancy models are tools with potential benefits in surgical training and patient education [1,2]. Most importantly, 3D models may facilitate surgical planning by allowing surgeons to assess tumor complexity as well as the relationship of the tumor to major anatomic structures [3]. The objective of this study was to evaluate this impact. Methods Imaging was obtained from an IRB approved, prospectively collected database of multiparametric magnetic resonance imaging (MRI) of renal masses. Ten cases eligible for elective partial nephrectomy were retrospectively selected. High-fidelity models were 3D printed in multiple colors based on T1 images (Fig. 1). Cases were reviewed by three attending surgeons and six senior residents with imaging alone and in addition to the 3D model. A standardized questionnaire was developed to capture the planned surgical approach and intraoperative technique in both sessions.

Fig. 2 Proportion Change in Operative Planning with 3D Model of Renal Mass Conclusion Utilization of 3D modeling may aid in pre-operative and intra-operative planning for both attending and resident surgeons. While 3D models with MR imaging is feasible, computed tomography (CT) imaging may provide additional anatomical information. Future study is required to prospectively assess the utility of models and pre-operative planning and intra-operative guidance. References [1] Mitsouras D, Liacouras P, Imanzadeh A, Giannopoulos AA, Cai T, Kumamaru KK, George E, Wake N, Caterson EJ, Pomahac B, Ho VB, Grant GT, Rybicki FJ. (2015) Medical 3D Printing for the Radiologist. Radiographics. 2015 Nov-Dec;35(7):1965–88. [2] Esses SJ, Berman P, Bloom AI, Sosna J. Clinical applications of physical 3D models derived from MDCT data and created by rapid prototyping. (2011) AJR Am J Roentgenol. 196(6):W683–8. [3] Rengier F, Mehndiratta A, von Tengg-Kobligk H, Zechmann CM, Unterhinninghofen R, Kauczor HU, Giesel FL. (2010) 3D printing based on imaging data: review of medical applications. Int J Comput Assist Radiol Surg. 5(4):335–41.

Fig. 1 3D printed tumor models. A, B) Axial and Sagittal MRI images. C) 3D projection D) 3D printed model Results Surgical approach was changed in 20 % of decisions, intraoperative considerations were changed in 40 % (Fig. 2). Thirty percent and 23 % of decisions in the attending and resident groups, respectively, were altered by the 3D model. Overall, every case was modified with this additional information. All participants reported that the models helped plan the surgical approach for partial nephrectomy. Most reported improved comprehension of anatomy and confidence in surgical plan. Half reported that the 3D printed model altered their surgical plan significantly. Due to use of T1 images, reconstruction of calyces and tertiary blood vessels were limited: 8 of the 9 participants desired more information regarding these structures.

Use of the new Integrated Table Motion for the Da Vinci Xi surgical system in abdominal surgical procedures L. Morelli1,2, S. Guadagni1, M. Palmeri1, G. Di Franco1, T. Simoncini3, A. Perutelli4, V. Cela3, C. Selli5, F. Francesca6, M. Cecchi7, P. Buccianti8, M. Anselmino9, G. Naldini10, C. Zirafa11, F. Mosca2, F. Melfi11 1 General surgery unit, Department of Oncology, transplantation and New technologies, Pisa, Italy 2 EndoCAS (Center of Computer Assisted Surgery), University of Pisa, Pisa, Italy 3 Division of Gynecology and Obstetrics, Maternal Fetal Department, Pisa, Italy 4 2nd Department of Obstetrics and Gynecology, Maternal Fetal Department, Pisa, Italy

123

Int J CARS 5

Urology unit, Department of Oncology, transplantation and New technologies, Pisa, Italy 6 2nd Division of Urology, Department of Oncology, transplantation and New technologies, Pisa, Italy 7 Urology unit, Azienda Ospedaliera Viareggio, Viareggio, Italy 8 General Surgery, Department of Gastroenterology and Infectious Diseases, Pisa, Italy 9 Bariatric Surgery, Department of Gastroenterology and Infectious Diseases, Pisa, Italy 10 Division of Proctological and perineal Surgery, Department of Gastroenterology and Infectious Diseases, Pisa, Italy 11 Multidisciplinary Center of Robotic Surgery, Azienda OspedalieroUniversitaria Pisana, Pisa, Italy

complications or need for conversion to laparoscopy or laparotomy. There were no ITM safety-related observations and no adverse events ITM or device-related. Conclusion This preliminary study demonstrated the efficiency of ITM for the da Vinci Xi Surgical System, which enabled patient repositioning without disrupting surgical workflow by allowing the surgeon to leave instruments and the scope docked to the patient. ITM has been shown to be safe, and no adverse events related to its use were reported. Further studies can be useful to demonstrate if ITM could enable procedures or part of procedures to be done robotically that would otherwise be difficult, and if ITM could improve operative efficiency by reducing surgical operative time (Fig. 1).

Keywords New technology  Da Vinci Xi  Table motion  Minimally invasive surgery Purpose Integrated Table Motion (ITM) for the da Vinci Xi Surgical System is a new feature comprising of a Trumpf TS7000dV Operating Table, that communicates wirelessly with the da Vinci Xi. The feature allows the surgical staff to reposition the patient without undocking the robot from the patient and without removing instruments from inside the abdomen. ITM has been specifically developed to improve multiquadrant robotic surgery. We herein present the first human use of this device across multiple surgical disciplines in the EU. Methods Between May and October 2015 one of the first human use of ITM was conducted in a post market study in the EU in which 30 cases across different specialties were prospectively enrolled. We included general surgery procedures, urological procedures and gynecological procedures. Variables examined included patient characteristics and intraoperative data. Primary end-points were: ITM efficacy, safety and efficiency. For these reasons we evaluated the number of table moves per case, duration of each table move, table positions attained, reasons for moving the table and the states of instruments and endoscope during table move (inserted or removed). We also evaluated the safety of ITM by recording occurrence of adverse events related to the use of ITM. Results Twelve patients underwent general surgery procedures (six cases of anterior rectal resection (ARR) with TME, 3 cases of right hemicolectomy, a case of subtotal gastrectomy, a case of hepatic resection and a case of ventral hernia repair. Gynecological procedures included five cases of hysterectomy, a case of repair of rectal prolapse and a case of uterine prolapse. Urological procedures included seven cases of prostatectomy, a case of nephrectomy, two cases of partial nephrectomy and a case of pyeloplasty. The mean ITM moves during the colorectal procedures was 3, while ITM was moved two times for repair of rectal prolapsed, three times during subtotal gastrectomy and only one time during liver resection and ventral hernia repair, resulting in 35 instances of table moves in 13 procedures. The mean ITM moves during the repair of uterine prolapse was 5, while ITM was moved on average 3.4 times during hysterectomy, resulting in 22 instances of table moves in 6 procedures. The mean ITM moves during prostatectomy was 3.1, while ITM was moved 2 times during partial and total nephrectomy and pyeloplasty, resulting in 30 instances of table moves in 11 procedures. Majority of moves ([ 70 %) took less than 2 min to complete. The primary reason for using ITM was to gain internal exposure in 82 moves (93 %). The endoscope was left inserted during 91–93 % of table movements, while the instruments were left inserted in 95–97 % of moves. No external collisions or other problems related to the operating table were noted. There were no ITM related surgical

123

Fig. 1 Plots of the table moves (Trendelenburg and Tilt) by specialty across 30 patients

Novel methods for data and image visualization and interaction in surgery using virtual and augmented reality H. G. Kenngott1, A. A. Preukschas1, L. Bettscheider1, S. Speidel2, M. Pfeiffer2, M. Huber2, M. Mu¨ller3, L. Maier-Hein3, H.- P. Meinzer3, B. Radeleff4, H.-U. Kauczor4, B. P. Mu¨ller1 1 Heidelberg University, General, Visceral and Transplantation Surgery, Heidelberg, Germany 2 Karlsruhe University, Karlsruhe Institute for Technology, Karlsruhe, Germany 3 German Cancer Research Center, Division of Medical and Biological Informatics, Heidelberg, Germany 4 Heidelberg University, Diagnostic and Interventional Radiology, Heidelberg, Germany Keywords Operation planning  Liver resection  3D-reconstruction  Human machine interface Purpose Medical imaging is essential for the diagnosis and therapy of patients across a broad spectrum of medical disciplines. Imaging data is required at different points of the surgical treatment process in different modes of presentation. Traditional ways of viewing medical imaging data (i.e., Picture Archiving and Communication System (PACS) workstations) in a stacked fashion removed from the patient may change with virtual and augmented reality (AR). We firstly present a new, interactive and immersive method of visualizing preoperative planning data with a 3D virtual reality framework using the head-mounted display (HMD) Oculus RiftTM (Oculus VR LLC, Menlo Park, CA, USA) and show feasibility, assess satisfaction and acceptance of the technology and evaluate its potential by medical professionals, Fig. 1. Secondly we present a mobile, real-time and point-of-care augmented reality system for medical diagnosis and therapy and evaluated it with regard to feasibility and accuracy in a pilot study involving phantom, animal, and human models, Fig. 2. Our aim was to evaluate the potential of virtual and augmented reality in the surgical treatment process.

Int J CARS

Fig. 1 3D-visualization as seen through the head-mounted display

Fig. 2 Mobile augmented reality Methods For virtual reality we selected a sample case of a colorectal cancer patient with multiple hepatic metastases planned for liver resection. We segmented the liver and gallbladder surface, the arterial, venous and portal venous vasculature, the bile ducts and the liver tumors with a defined security margin from a CT-scan using the arterial and venous phases. We used the Medical Imaging Interaction Toolkit (MITK, German Cancer Research Center, Heidelberg) for organ and vessel segmentation. Post-processing was carried out with MeshMixer (Autodesk Inc., San Rafael, CA, USA). We used the virtual reality HMD to visualize the intraoperative anatomical situation in the abdomen. By using the HMD the surgeon could access the three-dimensionally visualized upper abdomen, clinical patient data and the original CT images. We evaluated user satisfaction, acceptance and potential by an online questionnaire. For augmented reality computed tomography imaging was realized and a tablet computer was positioned above the patient and a semi-transparent 3D-representation of structures of interest were superimposed on top of the patient’s image resulting in augmented reality. Live camera images and the three-dimensional volume were registered by fiducial markers. Feasibility and accuracy were evaluated in a static model using the open source Heidelberg Laparoscopy Phantom (openHELP). The system was further analyzed in a porcine animal study. The reprojection error for both phantom and animal studies was defined as the average offset of the back-projected two-dimensional image points

and the manually defined points in the three-dimensional volume. Finally the setup was tested with a human volunteer to prove basic feasibility for clinical application. Results We evaluated the virtual reality system with attending surgeons (n = 13), resident surgeons (n = 34), medical students (n = 57), surgical nurses (n = 52) and non-medical staff (n = 22). In total 180 evaluations were performed. 89 % of the users were satisfied with the virtual reality scene, 90 % saw its potential to better evaluate complex surgical cases, 87 % found it useful in the training of medical students, 85 % in surgical training and 56 % in the training of nurses. 80 % saw high clinical potential of this technology. The AR-System was successfully realized in the phantom, animal and human model. In the phantom model of the 1380 analyzed AR-positions 83.9 % could be successfully realized. The reprojection error was 2.83 ± 2.68 mm. 95 % of the measurements were below 6.71 mm. In the animal model 79.3 % of the 690 analyzed AR-positions could be successfully realized. In the animal study the reprojection error was 3.52 ± 3.00 mm. 95 % of the measurements were below 9.49 mm. The reprojection error was significantly lower in the phantom model compared to the porcine model (P \ 0.001). At last augmented reality was successfully realized in clinical case. Conclusion Novel methods of image visualization and interaction are feasible, accurate and clinically practicable. Three-dimensional surgery planning and simulation in combination with immersive virtual reality can prove beneficial for complex liver resections, possibly improving patient care. It may also help younger surgeons to better understand the underlying anatomy of a surgical case and the reasoning behind the surgical decision-making processes. Secondly, mobile, real-time and point-of-care augmented reality systems for clinical purposes are feasible and accurate in a realistic experimental setting. Acknowledgement The current study was conducted within the setting of Research Training Group 1126: ‘‘Development of New Computer-Based Methods for the Future Workplace in Surgery’’ and the Collaborative Research Center 125: Cognition Guided Surgery, both funded by the German Research Foundation (DFG).

Prediction of head shape following spring cranioplasty: a case study A. Borghi1, N. Rodriguez Florez1, D. Dunaway2, O. Jeelani2, S. Schievano1 1 University College London, Institute of Child Health, London, Great Britain 2 Great Ormond Street Hospital, Craniofacial Unit, London, Great Britain Keywords Biomechanics  Spring cranioplasty  Finite element  Surgical planning Purpose Craniosynostosis consists of premature fusion (ossification) of one or more cranial sutures during infancy; it affects 1 in 1,700 live births [1]. The most common presentation is sagittal craniosynostosis (SC), which occurs when the sagittal suture fuses. SC causes craniofacial deformity and may cause functional problems as well as raised intracranial pressure. Conventional treatment for this condition involves surgical removal of the fused sagittal suture, placement of several cuts along the skull bones to allow for appropriate brain growth, and, following surgery, in some instances customized moulding helmet therapy for 4–6 months. The surgery usually takes 4–5 h and complications are mainly related to bleeding and need for a transfusion [2]. A less invasive procedure was pioneered at Great Ormond Street Hospital: a skull osteotomy in conjunction with metal distractors implanted in parasagittal position (springs) allows gradual moulding

123

Int J CARS of the skull to the desired shape (Fig. 1). This technique, called Spring Cranioplasty, has shortened considerably not only the time of surgery, but also the patient postoperative recovery. Spring Cranioplasty has proved to be a valid alternative to traditional more invasive surgeries, as it provides similar functional results with shorter operative time and lower blood loss; however, final aesthetic outcomes remain partially unpredictable. Computational modeling has the potential to become a powerful tool for surgical planning of craniofacial procedures: in this work, a computational model of spring cranioplasty was built and validated using clinical data from a patient, thus demonstrating feasibility of such methodology.

using measurements taken in theatre. Stress and displacement pattern were retrieved and analyzed: peak displacement occurred around the area of the spring placement, the skull underwent a complex stress pattern with the parietal bone undergoing bending due to the location of the springs (Fig. 2).

Fig. 2 Post-op skull stress pattern (left) and colour map showing differences between the predicted shape of the head and the 3D scan (right)

Fig. 1 Sample of skull after spring insertion and cranioplasty spring (in the insert) Methods A 5 month-old boy diagnosed with scaphocephaly (elongated head shape) was surgically treated with springs in March 2014 and springs were subsequently removed 6 months later. In theatre two springs were inserted, one in the anterior and one in the posterior position. During surgery, position and dimension of the osteotomies, type and model of the springs were recorded. A head surface scan was acquired in theatre right after the implantation, using a portable 3D scanner. Computed Tomography (CT) data were processed in order to create a 3D model of the patient head: the scans were imported into ScanIP (Simpleware) and hard (bone) and soft (sutures, scalp) tissues were segmented using grey value thresholding. 3D volumes were calculated for each type of tissue and tetrahedral volume mesh was created (244522 nodes and 858853 elements). Surgical osteotomies as well as bony notches were created using measurements collected during surgery. The model was then imported into Ansys Mechanical (Ansys Inc.) for simulation. Boundary conditions were applied: the bottom surface of the model was restrained in the vertical direction and a node was fully constrained to prevent rigid body movement. To simulate the effect of spring expansion, two predefined linear spring connections were applied in the locations equivalent to the slots performed during the surgery. The cranioplasty spring loading curves were retrieved experimentally: all springs are manufactured with an initial opening (foot plate to foot plate) of 60 mm and are crimped in theatre to a dimension of 20 mm for insertion. The anterior spring had a stiffness of 0.39 N/mm and after a single cycle of crimping would expand up to 57.3 mm. The posterior spring had a stiffness of 0.17 N/ mm and would expand back to 60 mm after release. Results The expansion of the sagittal springs was modeled, using a patient specific 3D geometry where surgical osteotomies were replicated,

123

The geometry of the outer scalp after spring expansion was extracted, post-processed using Meshmixer (Autodesk, California) and compared with the one retrieved in theatre: very good qualitative agreement was found. Quantitative comparison, showed a peak 5 mm difference, located in correspondence of the anterior and posterior fontanel (Fig. 2). Computational prediction of anterior and posterior spring opening compared well with the on-table measurements (3 % and 10 % difference respectively. The predicted post-surgery cranial index well correlated with that calculated from the 3D scan (78.7 % vs 80.3 %. Calculated post-operative head volume matched well the value calculated from the 3D scan (0.36 % difference). Conclusion A patient specific model of spring cranioplasty was created, using 3D data from medical imaging and surgical information gathered in theatre. The device used for the cranioplasty was tested in the lab and force vs opening release curves were used to implement a spring-like condition in ANSYS, which was able to replicate the behavior of the device in vivo. The spring expansion was modelled using literature material properties and the results show that the model accurately predicts the amount of expansion experienced in theatre andoverallvariation of skull dimensions. It was also able to predict the changes occurring on the head surface due to the internal distraction. Further refinement of this model, including the application of viscoelastic material properties, will allow modeling the time-dependent changes in skull dimensions. The final aim of this project is to enable surgical planning for spring cranioplasty and help optimize surgical parameters, such as spring location, model and size of the osteotomies, to ensure an already good functional outcome is matched by an equally positive aesthetic outcome. References [1] Fearon JA (2014) Evidence-based medicine: Craniosynostosis. Plast. Reconstr. Surg. 133:1261 [2] David LR, Plikaitis CM, Couture D, Glazier SS, Argenta LC (2010) Outcome analysis of our first 75 spring-assisted surgeries for scaphocephalyc. J Craniofacial Surg. 21(1): 3–9.

Anatomical landmark localization and ultrasound-CT image registration applied to surgery navigation of hepatectomy Z. Li1, P. Zhu1, M. Suzuki1 1 Hitachi,Ltd., Research & Development Group, Tokyo, Japan

Int J CARS Keywords Landmark localization  US-CT image registration  Ultrasound vessel extraction  Surgery navigation Purpose Image-guided navigation for oncological liver resections based on intraoperative ultrasound (US) image and preoperative CT image is important for operability decisions. In this work, a fully automatic surgery-navigation method which includes anatomical landmark localization, vessel extraction, and image registration of US-CT liver images, is proposed for Hepatectomy. Experiment results show that the proposed method can achieve fully automatic and robust registration of US-CT images and provide patient’s anatomical information using the locations of the detected landmark for guiding Hepatectomy operations. Methods The framework of the proposed method is illustrated in Fig. 1. First, an object detection method on the basis of Hough Forest [1] is proposed to classify and localize the right and left branch of portal vein (PB) from 3D US image and 3D CT image, respectively. A coarse-to-fine training and searching strategy is proposed for the Hough forest detector to efficiently and accurately localize the corresponding branch.

Second, a novel US vessel extraction method is proposed to extract portal vessel regions in the local 3D image which is cropped around the detected branch from the 3D US image. Center positions and diameters of the US vessels are estimated by applying curvature maxima searching [2] on cross-sectional intensity profiles of the US image. As a result, the vessel regions locate in intensity dips can be efficiently extracted. The searching procedures of the intensity dips are shown in Fig. 2. In the meantime, CT vessel regions around the vessel branch are extracted by using a region-growing-based method [3].

Fig. 2 Curvature searching on US intensity profile for vessel extraction Third, iterative closest point (ICP) [4] is adapted and modified for point set registration of vessel points extracted from the US-CT images. A robust wrong-pair rejection approach based on median absolute deviation is proposed for improving robustness and accuracy of the vessel point registration. In addition, the localization of the corresponding vessel branch provides an appropriate starting point for the point set registration, so an efficient and fully automatic US-CT registration can be achieved. The proposed method is implemented to a US system equipped with a probe position sensor. Once the 3D US-CT image registration is completed, real-time surgery navigation by indicating the same cross section as US image from CT image can be achieved. Moreover, patient anatomical information can also be achieved by using the landmark localization results. Fig. 1 The framework of the proposed method

123

Int J CARS Results Experiments on 15 US-CT image pairs, which were acquired in livertumor resection surgery, were conducted. First, localization errors of the anatomical landmark (here, PB) in US images and CT images were measured. In order to increase the amount of training data for the Hough Forest detector,for each image, 10 patterns of random translations between [-10, 10] mm and random rotations between [-15, 15] degrees around the landmark PB were added. The average localization errors of a 5-fold cross validation were 3.8 mm and 6.7 mm in US images and CT images, respectively. It can be concluded that the localization of the landmark PB can provide an appropriate initial position for the following registration. Moreover, the localization result of the landmark provides important anatomical information for the surgery navigation system. For evaluating the registration accuracy of the proposed method, corresponding liver vessel branches other than PB were annotated on the US-CT images by a surgeon. The average registration error measured on the annotated branches was 5.8 mm, and that of manual registration by the surgeon was 11.4 mm. Computational time including vessel branch localization, vessel extraction, and registration was less than 10 s on a system with Intel CoreTM i7 CPU 3.70GHz and 12-GB memory. It can be concluded that the proposed method can achieve a fully automatic and reliable registration for surgery navigation of Hepatectomy. Conclusion The proposed surgery navigation method can not only achieve automatic and robust registration of US-CT images but also provide the locations of anatomical landmark (here, PB) that is significant for guiding Hepatectomy operations. References [1] Juergen G et al. (2011) Hough forests for object detection, tracking, and action recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 11, pp. 2188–2202 [2] Miura N, Nagasaka A, Miyatake T (2007) Extraction of fingervein patterns using maximum curvature points in image profiles. IEICE Transactions on Information and Systems, vol. 90, no. 8, pp. 1185–1194. [3] Miura K et al. (2001) Hepatectomy simulation—its fundamentals and clinical application. MEDIX, vol. 35, pp. 9–14. [4] Besl PJ, McKay ND (1992) A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239–256.

Atlas-based pedicle trajectory prediction assessment and guidance of screw insertions

for

automatic

J. Goerres1, T. De Silva1, A. Uneri1, M. Ketcha1, S. Reaungamornrat1, S. Vogt2, G. Kleinszig2, J.-P. Wolinsky3, J. H. Siewerdsen1 1 Johns Hopkins University, Biomedical Engineering, Baltimore, United States 2 Siemens, Healthcare XP, Erlangen, Germany 3 The Johns Hopkins Hospital, Department of Neurosurgery, Baltimore, United States Keywords Pedicle screw  Vertebra registration  Trajectory prediction  Image-guided surgery Purpose Minimally invasive fixation of vertebral bodies requires percutaneous insertion of surgical screws. Commonly, these screws pass through the pedicle, a narrow corridor within the vertebral body

123

bounded by nerves and vessels. High precision is required to prevent breaches and to avoid damage to surrounding structures, particularly the adjacent vertebral canal that surrounds the spinal cord. Different approaches have been proposed to help the surgeon achieving accurate screw placement. Navigation systems can provide guidance within the context of intra-operative CT images and allow highly accurate placements through the pedicle [1]. Imaging systems with X-ray to CT registration of pedicle screw placements can be used to allow intra-operative quality assurance [3]. However, both concepts require a manual annotation of the target screw trajectory, which is time-consuming and can interrupt or complicate the surgical workflow. In this work, we present an approach for automatic determination of a transpedicle trajectory (that implicitly allows derivation of the most appropriate screw diameter and length). The algorithm utilizes an atlas-based registration approach and operates on a patient-specific preoperative 3D image (CT or MRI) in which the target vertebrae have been segmented. The approach therefore benefits from recent advances in automatic spine level detection and segmentation, which can be fully automatic and highly accurate [2]. Based on the segmentation, pedicle screw trajectories are automatically determined using knowledge annotated within the atlas. The computed trajectory can in turn facilitate more streamlined navigation and serve as a reference for automatic quality assurance of screw placement [3]. Methods With a segmentation of the vertebrae represented as a surface we attempt to detect screw entry points, the point of anterior-most extent of the screw trajectory, and the pedicle center. Atlas surfaces are registered in order to map prior annotations to the unknown vertebra surface. An atlas of 20 spine CT images was built using SpineWeb (open datasets 2 and 4 on http://spineweb.digitalimaginggroup.ca) in which the points were annotated. The work reported below considers the L3 vertebra binary segmentation extracted and transformed in a uniformly distributed surface representation Given a closed input surface, a non-rigid coherent point drift (CPD) registration [4] is applied to deform atlas surfaces into the input surface. The resulting deformed surfaces are then used to map the point annotations of the original atlases into their input-similar deformation through a common representation in mean value coordinates [5]. This representation approach allows interpolation of interior and exterior positions of a surface using a linear combination of vertex locations. A function is used to transform into mean value coordinates by determining the vertex weight coefficients and its inverse is used to transform back to Cartesian coordinates by applying the linear combination. In this work, this function is used to transfer the annotations into mean value coordinates of the atlas surface and the inverse function is used to transfer back using the deformed atlases with input-similar surface. The final annotation predictions for the input surface are then obtained by computing the mean locations of the transferred atlas annotations. A leave-one-out cross-validation is performed to determine the prediction errors of the annotation points using Euclidean distances. Additionally, the trajectories defined by the entry point and anterior location are assessed regarding their distance to the spinal canal in the pedicle. As shown in Figure 1a, the pedicle center is projected on the trajectory and from there, equidistant points are distributed along the trajectory within the pedicle (± 5 mm from the pedicle center). For each sample point, the shortest Euclidean distance to manually labelled vertices, which define the spinal canal, is computed. The same procedure is performed for the reference trajectory and is used to determine pointwise corresponding distance distributions between the sample points. Moreover, the registration is validated by computing Dice coefficients.

Int J CARS b Fig. 1 (a): For evaluation, the pedicle center is projected on the trajectory and equidistant sample points are used to determine distances to the spinal canal. (b,c): Lumbar vertebra L3 (white surface) with expert reference (yellow line), planning predictions (black lines), and the mean of these predictions, which we take as the proposed planned trajectory (cyan line) Results An example of automatically derived screw trajectories in L3 is shown in Fig. 1(b,c). The prediction errors were 1.10 ± 0.50 mm (mean ± standard deviation) for the pedicle center, 2.52 ± 1.12 mm for the screw entry point, and 3.47 ± 1.84 mm for the anterior location. Figure 2a shows the distance deviation of the predicted trajectory to its reference. Negative values indicate proximity to the spinal canal whereas positive values mean approaching the lateral cortex. The boxplot range inherently depends on the parallelism of reference and trajectory w.r.t. the spinal canal, because a similar error at all sample points implies a consistent deviation along the trajectory in the pedicle. For 70 % of the trajectories, a deviation to reference of less than 1.5 mm was achieved for all sample points in the pedicle. In Fig. 2b, the Dice coefficients of all non-rigid registrations are depicted in a boxplot and were between 0.91 and 0.96.

Fig. 2 (a): Distance deviation between trajectory and reference to the spinal canal at several sample points described in Fig. 1a. (b): Dice coefficients demonstrating the quality of the non-rigid atlas registrations

123

Int J CARS Conclusion We have presented a new method for automatic determination of pedicle screw trajectories, including identification of most suitable screw diameter (derived from the width of the pedicle neck computed in the registration process) and screw length (from the length of the trajectory from entry to anterior-most point). The approach only requires segmentation of the preoperative CT (or MR) and therefore allows redefinition of manual annotations in order to adjust the approach to a surgeon’s trajectory preference or to new surgery techniques. The method can facilitate or replace manual trajectory definition for image-guided navigation in percutaneous spinal fixation and intraoperative quality assurance. With potentially broader impact beyond spinal navigation, and since intraoperative X-ray images can be used to determine the 3D location of pedicle screws in preoperative CT data [3], the presented approach can also provide the necessary trajectory reference for quality assurance of pedicle screw placements. References [1] Houten JK, Nasser R, Baxi N ‘‘Clinical Assessment of Percutaneous Lumbar Pedicle Screw Placement Using the O-Arm Multidimensional Surgical Imaging System,’’ Neurosurgery, vol. 70, no. 4, pp. 990–995, 2012. [2] Ibragimov B, Korez R, Likar B, Pernusˇ F, Vrtovec T ‘‘Interpolation-Based Detection of Lumbar Vertebrae in CT Spine Images,’’ Recent Adv. Comput. Methods Clin. Appl. Spine Imaging, pp. 73–84, 2015. [3] Uneri A, Silva TD, Stayman JW, Kleinszig G, Vogt S, Khanna AJ, Gokaslan ZL, Wolinsky JP, and Siewerdsen JH ‘‘Knowncomponent 3D-2D registration for quality assurance of spine surgery pedicle screw placement,’’ Phys. Med. Biol., vol. 60, no. 20, p. 8007, 2015. [4] Myronenko A, Song X ‘‘Point Set Registration: Coherent Point Drift,’’ IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 12, pp. 2262–2275, 2010. [5] Ju T, Schaefer S, Warren J ‘‘Mean Value Coordinates for Closed Triangular Meshes,’’ in ACM SIGGRAPH 2005 Papers, 2005, pp. 561–566.

Learning curves in image guided spinal surgery: a human factors analysis showing surgeon expertise decreases flow disruptions D. Drazin1, K. Catchpole1, R. Pashman1, J. P. Johnson1, T. Kim1 Cedars-Sinai Medical Center, The Spine Center, Los Angeles, United States 1

Keywords Image guided spinal surgery  Learning curve  Human factors  Flow disruptions Purpose The newest generation of navigation technology—intraoperative computed tomography image-guided navigation (CT-IGN) with the mobile O-armTM scanner—has increased imaging resolution, leading to improved accuracy of instrumentation placement in the spine. However, applying navigation technology can increase the complexity of the surgical workflow, and lead to intraoperative disruptions that potentially effect patient safety and operative performance. Human factors and systems analysis approaches have been used to study the complex interplay between the surgeon, the other team members team, and the various technologies in other surgeries. The goal of this study was to apply the principles of human factors research and workflow systems analysis to CT-IGN spine surgery in order to better understand inefficiencies, workflow disruptions and identify potential causes of suboptimal outcomes.

123

Methods Using a previously validated technique, a single observer was trained to identify and record flow disruptions (FD) in 38 spinal surgeries that utilized CT-IGN with the O-armTM. FDs are defined as ‘‘deviations from the natural progression of a procedure that potentially compromise safety or efficiency,’’ and have been linked to process problems, increased errors, and near misses. The workflow of the CT-IGN spine case was timed, mapped, and all individuals involved in the spine surgery were tracked. The spine navigation surgery was divided into three main phases: pre-imaging, CT image acquisition, and instrumentation/screw delivery. FDs were organized into 8 categories: communication breakdown, coordination delays, environmental distractions, surgeon decision making disruptions, training disruptions, equipment issues, environmental factors and patient factors. The observer also recorded the experience level (expert/novice) of the team (surgeon, navigation technician, scrub nurse, radiology technician). Results Researchers observed 38 consecutive CT-IGN spine cases in the cervical, thoracic and lumbar spine. Altogether a total of 530 FDs were observed, with a mean of 14.7/case (95 % CI 11.93–17.47). The top five FD categories that were found to significantly impede CTIGN workflow in ranked order were: Coordination delays, equipment issues, environmental interruptions, surgical decision making disruptions and training disruptions Surgeon expertise ([ 50 navigated cases/year) and navigation technician expertise ([ 6 years of navigation experience) independently predict flow disruption rates. Using flow disruption rates during the CT-image acquisition phase, a linear regression suggests a baseline FD rate of 13.8 FD/Hour, reduced by 5.91 FD/hr with an expert surgeon (p = 0.0001), by 6.47 FD/hr with an expert navigation tech (p \ 0.0001), and by 8.38 FD/hr if both are experts (p = 0.0063). The most FDs were encountered during screw placement, with 38 % attributed to coordination delays, 30 % to equipment issues, and another 30 % shared evenly between training disruptions, external environmental interruptions and surgeon decision making disruptions. Coordination FDs occurred mostly in the pre-imaging and CT image acquisition phases, while equipment problems predominated during screw placement. Conclusion This study is the first of its kind to apply human factors analysis to CT-IGN spine surgery. Complexities associated with the intraoperative workflow in navigation spine surgery have lead to a high prevalence of delays, disruptions and surgeon dissatisfaction. Having a higher level of navigation expertise with both the Surgeon and navigation technician statistically reduces the rates of FD occurrence. Additionally, we found that coordination delays (which underscore how CT-IGN is a ‘‘multi-systems, multi-persons’’ process), and navigation equipment issues dominate the FDs encountered by the surgical team. Objectively describing and organizing the various FDs and their impact on the overall workflow are the initial key steps to understanding the problems associated with spinal navigation. Understanding and applying our findings to the training of the surgical team and to correct other problem areas will greatly improve the workflow, efficiency and delivery of navigation technology to consistently and effectively deliver CT-IGN in spinal surgery.

Segmented near infrared light image superimposition on visible light endoscopic video for gall bladder surgery A. Kumar1, S.- W. Huang1, Y.- Y. Wang1,2, K.- C. Liu1, W.- C. Hung1, Y.- C. Lee1 1 Chang Bing Show Chwan Memorial Hospital, IRCAD-Taiwan, Medical Imaging, Changhua, Taiwan, Province Of China 2 National Changhua University of Education, Changhua, Taiwan, Province of China

Int J CARS Keywords Laparoscopic cholecystectomy  Indocyanine dye  Augmented reality  Near infrared endoscopy Purpose Laparoscopic surgery of gall bladder, laparoscopic cholecystectomy (LC), is one of the most common surgeries. One of the most challenging steps in that surgery is the identification of biliary tract. Such structures are important to identify as its inadvertent excision may lead to poor outcome of the surgery. Although a well experienced surgeon may be able to identify those structures without much difficulty, it is a challenge to less experienced surgeon. Iatrogenic bile duct injuries are serious complications with patient morbidity. Recently, indocyanine green (ICG), a contrast agent used for diagnostic purposes such as ophthalmologic angiography and monitoring liver perfusion has been found to be promising in visualizing the biliary tract during laparoscopic surgery [1]. The contrast is injected into the patient before the procedure and the relevant organ is visualized with near infra-red (NIR) light of an endoscope. Few studies have reported use of such technique during the laparoscopic gall bladder surgery which helped in visualizing the biliary tract. However, it is difficult to perform the whole surgery under NIR. Therefore, during such surgery, surgeons need to swap multiple times between NIR and visible light (VL) that causes inconvenience to the surgeons as well as an increase in the duration of the surgery. This study presents a method which would reduce the number of times NIR and WL lights are swapped and thus reducing the duration of surgery and inconvenience to the surgeons. The method uses computer vision algorithms of registration, feature detection and tracking to superimpose the segmented NIR image over VL video image. To our knowledge, no study has been reported which address the issue. Methods A recorded laparoscopic video was used in the development of the method. The endoscope has both NIR and VL camera in the same system. However, NIR and VL cameras have different location inside the endoscope, therefore the images (Fig. 1) produced by them need to be aligned for further steps in the system. A mutual information based multimodal image registration technique was applied to register the WL and IR images [2]. The region of interest in the registered NIR image was segmented using a threshold technique in its blue channel in the RGB image. The intensity range in the blue channel was divided into three equal segments of high, medium and low intensity. The pixels which fall in the high intensity segment was selected as the area of interest. The transformation matrix between the first VL video frame and the subsequent VL frame was calculated with a feature based registration technique. Characteristic feature points in the first frame (F1) of the video sequence and the subsequent frame (Fn) were detected using the speeded-up robust feature (SURF) detection algorithm [3]. The detected interest points was assigned feature point descriptor known as binary string feature point descriptor (BRIEF) [4]. The descriptors in two consecutive frames were matched with Hamming distance criteria The outliers among the matched features were removed with RANdom Sample Consensus (RANSAC) [5]. The matched feature points on two consecutive video frames were used to calculate an affine transformation matrix by solving a set of linear equations using a least square minimization method. The matrix was applied to the edge image of the segmented NIR image to align it to the frame Fn. The transformed edge image was superimposed on the Fn and visualized.

Fig. 1 (Left) Image with visible light (VL). (Right) Image with near infrared light (NIR) Results The method was applied on 15 video segments each with more than 5 min of duration. A C ++ with CUDA based software for the system was run in a computer with Intel CoreTM i7 960 @3.20 GHz, 6.00RAM 64 bit Windows 7 and graphics card of NVidia TESLA C2075. The software would use the libraries of VTK, OpenCV and ITK. An example image with overlapped NIR segmented edge on the VL image is shown in Fig. 2.

Fig. 2 Superimposed edges of segmented registered NIR image on VL image Conclusion A method to superimpose NIR image to the VL laparoscopic video was developed. The method may help surgeons in reducing the time of the laparoscopic gall bladder surgery. The method will be evaluated for the accuracy of the superimposed area registration and contribution in reduction in time taken for the surgery. References [1] Osayi SN et al. (2015) Near-infrared fluorescent cholangiography facilitates identification of biliary anatomy during laparoscopic cholecystectomy. Surgical endoscopy, 2015. 29(2): p. 368–375 [2] Pluim JP, Maintz JA, Viergever M (2003) Mutual-informationbased registration of medical images: a survey. Medical Imaging, IEEE Transactions on, 2003. 22(8): p. 986–1004 [3] Bay H, Tuytelaars T, Van Gool L (2006) Surf: Speeded up robust features, in Computer vision-ECCV 2006. 2006, Springer. p. 404–417 [4] Calonder M et al. (2010) Brief: Binary robust independent elementary features. Computer Vision-ECCV 2010, 2010: p. 778–792 [5] Kim J-H et al. (2012) Tracking by detection for interactive image augmentation in laparoscopy, in Biomedical Image Registration. 2012, Springer. p. 246–255

123

Int J CARS A surgical navigation system for laparoscopic pelvic lymph node dissection: registration and evaluation on a patient-specific phantom L. Ma1, K. Qian1, J. Wang1, Y. Tomioka1, H. Kiyomatsu1, E. Kobayashi1, I. Sakuma1 1 The University of Tokyo, Tokyo, Japan Keywords Navigation system  Pelvic lymph node dissection  Free-hand ultrasound  Stereo tracking Purpose Nowadays, in order to prevent the lymphatic metastases and cancer recurrence, pelvic lymph node dissection (PLND) is generally performed after the standard cancer dissection in the treatment of some pelvic cancers, such as rectal cancer and prostate cancer. However, PLND is a delicate procedure during which surgeons should localize the pelvic arteries covered by other tissue and then remove the lymph node surrounding the pelvic arteries. Therefore, if the surgeons cannot localize the arteries correctly and quickly during the procedure, the incidence of bleeding and overall surgical time may become much higher. In this paper, a navigation system is developed for the delicate PLND procedure. In this navigation system, a free-hand ultrasound probe without using external sensors is proposed to detect pelvic arteries intra-operatively [1], and this probe is tracked by detecting the position of dot array markers attached to the probe using stereo endoscope [2]. To relate the high resolution pre-operative CT image to intra-operative scene, a robust rigid registration method based on Gaussian mixture model (GMM) is used to register the centerlines extracted from the ultrasound image and CT image together. In order to evaluate the navigation system we proposed, validation experiment is also carried out using patient-specific phantom. Methods By using the free-hand ultrasound probe we proposed, a sequence of 2D ultrasound images {U1, U2, …, Un} and their corresponding position transformations {T1, T2, …, Tn} in respect to the stereo laparoscope can be generated after probe scanning. Here, Ti is a 4x4 matrix which is calculated using the dot array markers shown in Fig. 1 under stereo endoscopic [2]. In order to extract 3D centerline from the 2D US images, for an US image Ui, we firstly use the vessel enhancement filter to process the image, and then we select the regions whose area is similar to artery. Afterward, the morphological operations dilation/erosion are performed to remove noise, and the centre point can be acquired by fitting ellipse to the final contours. Finally, the 3D position of the centre point in endoscopic coordinate system can be obtained using the transformations Ti. A 3D centerline can be generated after processing all the 2D US images.

Fig. 1 Dot array markers attached free-hand ultrasound probe

123

In order to relate the pre-operative CT image to the intraoperative scene, we register the artery centerline extracted from CT images to the centerline from US images. Since the probe can only acquire images of common iliac arteries due to the restricted space in pelvis, we first register the upper pelvic arteries together rigidly and then transform the whole centerline from CT using the rigid transformation generated from the registration. In this registration, we employ the Gaussian mixture model (GMM) based rigid registration method whose key idea is to represent the input point clouds using GMM [3]. A validation experiment is carried out using a patient-specific phantom to evaluate the proposed navigation system. To build a patient-specific phantom, we make an artery model using urethane gel material based on the 3D model segmented from a patient. Results In the phantom experiment, we extracted the centerline of the common iliac artery from the US image sequence, and then registered the centerline of common iliac artery extracted from CT image to the centerline extracted from US. According to the registration results, the RMS of the rigid registration between the two centerlines was 1.29 mm. In order to relate the whole pelvic arteries in CT image to intra-operative scene, we transformed the centerline of the whole artery from CT using the rigid transformation generated in the registration above. The results of the registration in this validation experiment are shown in Fig. 2.

Fig. 2 The results of patient specific phantom experiment Conclusion We developed and implemented a surgical navigation system for laparoscopic pelvic lymph node dissection using a free-hand ultrasound probe tracked using internal dot array markers. 3D centerlines were extracted from intra-operative ultrasound images and CT image and registered together to relate the pre-operative CT image to the intra-operative scene. The registration method used in this paper was the Gaussian mixture model based rigid registration. Finally, the proposed navigation system was evaluated using a patient-specific phantom. In future work, we will perform the validation experiment on animals to prove its practicality in in vivo environment. Meanwhile, we will research the deformation of the pelvic arteries during the surgery and do our endeavor to develop a more accurate navigation system for PLND. References [1] Sen K, Wang J, Ando T, Kiyomatsu H, Kobayashi E, Sakuma I (2014) Navigation system for endoscopic lateral lymph node dissection. 23rd Annual Congress of Japan Society of Computer Aided Surgery: 180–181.

Int J CARS

[2]

[3]

Wang J, Kobayashi E, Sakuma I (2015) Coarse-to-fine dot array marker detection with accurate edge localization for stereo visual tracking. Biomedical Signal Processing and Control 15: 49–59. Jian B, Vemuri BC (2011) Robust point set registration using gaussian mixture models. Pattern Analysis and Machine Intelligence, IEEE Transactions on 33.8: 1633–1645.

The analysis of affecting factors on the application accuracy for frameless neuro-navigation Q. Li1, M. Guthikonda1, S. Mittal1 1 Wayne State University, Medical School, Neurological Surgery Dept., Detroit, United States Keywords Application accuracy  Surface match registration  Point to point registration  Neuronavigation Purpose The development of image guided neuro-navigation technology over the past decade has greatly improved clinical utilization of neuronavigation system for routine neurosurgical procedures. The application accuracy of the neuro-navigation system is the key issue for a successful neuro-navigation. Several affecting factors on the application accuracy of the neuro-navigation have been studied and published.[2-5]The purpose of this study was to evaluate several different factors that might affect the application accuracy of frameless neuro-navigation inside OR set up environment and build up a basic guild line for clinical neuro-navigation. Methods This study was performed using a phantom. This phantom was mounted with seven adhesive fiducial markers that randomly distributed on the surface (IZI Medical Products, Owings Mills, MD, USA). The two target points were set up at anterior skull base and posterior skull base using implantable screw markers (Fisher-Leibinger, Freiburg, Germany).(Fig. 1) A routine CT scan was done following the stereotactic protocol (Pixel Size: 0.455 9 0.455, Slice Thickness 1 mm). Brain-lab neuro-navigation system (Brainlab AG, Kapellenstrabe 12, 85622 Feldkirchen, Germany) was used for this study. Two registration methods were used for this study, surface match registration and point to point registration. The PRF was also set up two positions, one was set up with targets in 15 cm (Fig. 2) and another set up was 30 cm. The tip of each target was digitized and the coordinates were recorded before the test from CT scan. During the test, each target was digitized 10 times at each set up and was compared with pre-test recorded reference. The root mean square (RMS) was calculated to show the difference between the actual points and the measured points for each set-up. The differences from the reference points were used as the deviation from the ‘‘true point ‘‘.

Fig. 1 implantable screw markers (Fisher-Leibinger, Freiburg, Germany)

Fig. 2 Digitizing the posterior target and recorded as reference point before the test Sum of Errors = SQRT((x1-x2)2 + (y1-y2)2 + (z1-z2)2)

123

Int J CARS Results The results from each set-up are shown below: 1. The sum of errors when PRF located at 15 cm with targets [5] Surface Registration

Point Registration

Anterior Target

1.48 ± 0.12 mm

1.1 ± 0.2 mm

Posterior Target

2.5 ± 0.2 mm

1.76 ± 0.2 mm

2. The sum of errors when PRF located at 30 cm with targets Surface Registration

Point Registration

Anterior Target

3.48 ± 0.12 mm

1.7 ± 0.2 mm

Posterior Target

5.18 ± 0.3 mm

1.3 ± 0.2 mm

experimental error assessment quantifying registration methods and clinically influencing factors. Neurosurg Rev 34:217–228. 2011. Wang MN, Song ZJ: Properties of the target registration error for surface matching in neuronavigation. Computer Aided Surgery 16(4):161–169. 2011.

Towards uncertainty-aware navigation

auditory

display

for

surgical

D. Black1,2,3, B. Kocev1,2,3, H. Meine1,3, A. Nabavi4, R. Kikinis1,3 1 University of Bremen, Bremen, Germany 2 Jacobs University, Bremen, Germany 3 Fraunhofer MEVIS, Bremen, Germany 4 INI International Neuroscience Institute, Hannover, Germany Keywords Soft-tissue navigation  Auditory navigation  Uncertain navigation information  Auditory display

The above results showed that when PRF was located at 15 cm with target points, the two registration methods displayed no significant difference although the anterior target had better results at the surface matching registration group because that registration only focused on the face. When PRF located at 30 cm with target points, the system application accuracy reduced on surface matching registration method. Each target point increased more than 2 mm errors. But in point to point registration method, there was less than 1 mm difference. Conclusion From our experimental results presented, the following points can be concluded: 1. During routine neuro-navigation, the position of patient reference frame must be located close to the surgical target point. Especially using surface matching registration method. The application accuracy was the worst in the group of the PRF in 30 cm with target points and using surface matching registration method for the posterior skull base target point. 2. Compare two registration methods, point to point registration provided better results. There is no significant difference in different the location. When using surface matching registration during the navigation, there are two distances directly related with the application accuracy of neuro-navigation. One is the PRF with surgical target point and another is the surgical target point with face. 3. No matter what kind of registration methods you use during neuro-navigation[1], visual verification is always the only way to check the application accuracy in neuro-navigation. References [1] Agrawal D., Steinbok P. Fiducials: Achilles’ heel of ImageGuided Neurosurgery: An Attempt at Indigenization and Improvement. Clinical Neurosurgery Volume 56, 80–83. 2009. [2] Li Q., Zamorano L., Jiang Z., Diaz F.: The Application Accuracy of the Frameless Implantable Marker System and Analysis of Relating Affecting Factors: Lecture Notes in Computer Science 1496. Eds by William M. Wells, Alan Cohchester and Scott Delp, Page 253–260, 1998. [3] Li Q., Zamorano L., Jiang Z., Gong J., Pandya A., Perez R., Diaz F.: Effect of Optical Digitizer Selection on the application accuracy of a Surgical Localization System—A Quantitative Comparison between the OPTOTRAK and FlashPoint Tracking system. Computer Aided Surgery 4:314–321. 1999. [4] Paraskevopoulos D., Unterberg A., Metzner R., Dreyhaupt J., Eggers G., Wirtz CR.: Comparative study of application accuracy of two frameless neuronavigation systems:

123

Purpose In image-guided navigated interventions, information is often placed on screens to aid the operator in coordinating the placement of a tool with a pre-interventional plan. However, such methods require an operator to repeatedly shift the view from a patient. Auditory display for navigation (e.g., during ablation needle placement [1], path following [2], volume resection [4], etc.) offers suitable means for conveying numerical information using sound, permitting the operator to keep attention on the patient. The numerical navigation information, however, is uncertain due to various sources of potential errors, including from soft-tissue motion estimation processes [3] and tool-tracking hardware. We enhance an existing auditory display approach for soft-tissue navigation to consider this uncertainty in the numerical information. The operator should not only hear navigation cues towards relevant structures, but also receive information on the quality and reliability of such navigation parameters. The consideration of uncertainty in the operating room is, in general, an exciting area of exploration in the usability of new navigation systems, and its transmission with audio significantly offloads reliance on traditional visual displays. Methods Our uncertainty-aware auditory display encodes the numerical navigation information and the uncertainty present therein during a twodimensional (in-plane) targeting task. This mimics an example task of finding a skin insertion point during needle ablation, for which an auditory display has been previously developed [1]. The numerical information is in the form of ‘‘Euclidean distance to target.’’ We assume that the uncertainty in the distance is estimated and given as part of the input to our auditory display mechanism. Furthermore, we assume the standard linear model for the distance information: dðtÞ ¼ yðtÞ þ eðtÞ where d(t) is the input distance at time t, y(t) is the true (but unknown) distance at time t, and e(t) is the error at time t. The error e(t) is assumed to be Gaussian-distributed with zero mean. Hence, d(t) is also Gaussian-distributed, i.e., d(t) *N (l(t), r(t)). Therefore, our primary task is to use an auditory display to encode the first two moments (l(t), and r(t)) of the Gaussian distribution representing d(t). For that purpose, we create a single base tone and use two independent axes to relay the mean l(t). Mean distance along the y axis is represented by a reference pitch and a moving pitch played alternately at an inter-onset interval of 125 ms. When the tool moves up and down along the y axis, the moving pitch (261 to 1046 Hz quantized to a C-major scale) is brought closer or further away from the reference pitch (523 Hz). Mean distance along the x axis is mapped to the

Int J CARS sound’s stereo position; positive distances are heard in the left ear (target left of tool) and negative distances in the right ear (target right of tool). When the tool reaches the target, the sound is heard centered in both ears, and there is no deviation between moving and reference pitches (see Fig. 1).

Fig. 1 Auditory method to find target (green): Movements in y axis are heard by comparing alternating tones, while movements in x are encoded as position in stereo For encoding the variance r(t), we developed two alternative auditory methods based on different psychoacoustic principles. The first method maps the variance to the frequency modulation of the base sound by adding a vibrato component. This frequency modulation creates a wobbling tone to mimic an unstable state. At an uncertainty of zero, no modulation occurs. When the uncertainty reaches a maximum threshold value, a sine wave with a frequency of 12 Hz and amplitude of 4 is added to the base frequency. In this case, for example, a base frequency of 523 would modulate between 519 and 527 Hz at a rate of 12 Hz. The second method uses white noise audio signal to encode the variance r(t). The first moment of the distribution out of which the white noise audio signal is generated is set to zero, while its second moment is set equal to the variance r(t) (scaled between 0 and 1). This layer of white noise is added to the base signal, with amplitude ranging from silent at variance 0 and -6 dBFS at maximum variance. The motivation for implementing an audio noise signal is to mask the base signal, giving the operator increasingly perceptual uncertainty based on increasing variance. Results We performed a preliminary qualitative evaluation of the auditory display with medical imaging professionals, using a mouse pointer on a screen and requiring them to move the cursor to a target location by following only auditory navigation information. The 2D target was visually hidden from the users. This task simulates in-plane navigation of an ablation needle on the patient skin surface. Although still in an early phase, comments regarding the uncertainty-aware auditory display are promising, with results showing that users were able to perceive the level of uncertainty present in the navigation information. In addition, the auditory display method for blind target finding was well received, requiring only a short training time of less than five minutes. Conclusion We introduced a novel uncertainty-aware auditory display mechanism for interventional procedures, which contributes towards increasing awareness of the uncertainty present in numerical navigation

information. Ideally, this could increase the safety of critical navigation procedures. We retain the advantage of the auditory display that information is conveyed without the need for shifting the gaze away from the patient. In future work, we will evaluate both alternative methods for encoding the variance. Moreover, we will consider to which degree operators require uncertainty awareness, whether a simple binary signal might suffice, or whether a finer quantization is desired. In addition, several auditory display methods will be compared in an elaborate fashion. References [1] Black, D., Al Issawi, J., Hansen, C., Rieder, C., Hahn, H. K. Auditory Support for Navigated Radiofrequency Ablation. In Proc. Computer-und Roboterassistierte Chirurgie (CURAC) 2013, Innsbruck, Austria, pp 30–33. [2] Hansen, C., Black, D., Lange, C., Rieber, C., Lamade´, W., Donati, M., Oldhafer, K., and Hahn, H. Auditory support for resection guidance in navigated liver surgery. International Journal of Medical Robotics and Computer Assisted Surgery, 9(1) 36–42 (2013) [3] Kocev, B., Georgii, J., Linsen, L., Hahn, H. K. Information Fusion for Real-time Motion Estimation in Image-guided Breast Biopsy Navigation. In Proc. of Workshop on Virtual Reality Interaction and Physical Simulation (VRIPHYS) 2014, Bender, J., Duriez, C., Jaillet, F., Zachmann, G., Eds. Eurographics Association, pp 87–98. 10.2312/vriphys.20141227 [4] Woerdeman, P., Willems, P., Noordmans, H., and van der Sprenkel, J. Auditory feedback during frameless image-guided surgery in a phantom model and initial clinical experience, Journal of Neurosurgery, 110(2) 257- 262 (2009)

Automatic registration and error colormaps for navigated bone tumor surgery using intraoperative cone-beam CT M. Arkhangorodsky1, J. Qiu1, M. Daly1, P. Nayak1,2, R. Weersink1, H. Chan1, D. Jaffray1, J. Irish1, P. Ferguson2, J. Wunder2 1 Princess Margaret Cancer Centre, Guided Therapeutics (GTx) Program, Toronto, Canada 2 Mount Sinai Hospital, Division of Musculoskeletal Oncology, Toronto, Canada Keywords Surgical navigation  Automatic registration  Registration error  Intraoperative imaging Purpose In the workflow of image-guided surgery, registration can be a timeconsuming and frustrating experience for the surgeon. Choosing anatomical points in relevant regions can force exposure of more bone surface than otherwise necessary to perform the surgery. These challenges motivated the development of an anatomy-agnostic registration technique which leverages on optical tracker technology and intraoperative cone-beam CT imaging to eliminate the need for surgeons to select anatomical points in the image and patient space. The purpose of this clinical study is to validate an automatic registration technique against anatomical point-based registration using target registration error maps. Methods A pilot study involving 20 patients with benign osteochondral tumors of the extremity was designed to evaluate workflow and logistics of a novel surgical planning and navigation platform using intraoperative cone-beam CT (CBCT) imaging. In order to achieve fast and reliable tracker-to-image registration in the OR, custom metal-free optical tracking tools were designed out of Ultem plastic (polyetherimide).

123

Int J CARS As shown in Fig. 1(a), prior to CBCT imaging, a dynamic reference base is inserted and a pair of Ultem tools are attached to the patient’s skin using a sterile adhesive (Tegaderm) in a configuration that surrounds the lesion. The optical spheres act as surrogates for anatomic fiducial points, and provide a flexible mechanism of generating ‘anatomy-agnostic’ registration.

Fig. 1 (a) Intraoperative cone-beam CT imaging (Artis Zeego, Siemens Healthcare) during extremity bone tumor surgery using custom metal-free registration markers. (b) In-house surgical navigation system using optical tracking of planar cutting tools (e.g., osteotomes, saws) for real-time resection guidance Using a three-dimensional extension of the Hough transform algorithm, the centers of the optical spheres are localized within the 3D CBCT volume to be used as image landmarks for registration, and the corresponding tracker landmarks are captured using an NDI Polaris infrared camera. This process eliminates the need for surgeons to manually localize each fiducial point in order to reduce both OR time and fiducial localization error (FLE). For comparison, anatomic landmarks were also chosen in the image for manual point-based registration. The selected anatomic landmarks were constrained by the size of the surgical lesion created by the automatic registration technique. Two principal measures of error from both marker-based and anatomic-based registrations were obtained for each case: fiducial registration error (FRE) and target registration error (TRE). FRE is a single value which measures how well fiducials align, while TRE is a set of values that measure the effective navigation uncertainty calculated at every point in the image. As shown in Fig. 1(b), accurate registration enables the in-house navigation system to precisely track planar surgical instruments (e.g., osteotomes, saws) with real-time distance/pitch/roll indicators relative to planned planes as well as virtual bone clipping. Results To date, four patients with lower-extremity bone tumors have been included in the study (3 distal femur, 1 distal tibia/fibula). The placement of Ultem markers introduced minimal CT artifact, allowing them to be positioned in convenient configurations in close proximity to the surgical field. The mean FRE value over four patient registrations using the metal-free reference markers was 0.97 ± 0.23[standard deviation] mm. Representative examples of the TRE distribution across the bone volume are shown in Fig. 2, for both marker-based (Fig. 2a) and anatomy-based (Fig. 2b) paired-point registrations.

123

Fig. 2 Colormap representations of target registration error (TRE) distributions [mm], where blue points represent fiducial landmarks. (a) TRE distribution obtained using anatomical landmarks. (b) TRE distribution obtained using intraoperative registration markers For four cases, the mean TRE calculated over the entire surface of bone within the CBCT image was 0.99 ± 0.28 mm for marker-based registration, compared to 2.54 ± 1.54 mm for anatomy-based. The percentage of surface area encompassed by the 1 mm TRE zone was 60.0 % for marker-based, compared to 32.1 % for anatomy-based. The time required to perform the complete process of intraoperative CBCT imaging and tracker registration is currently *5–10 min, with further streamlining in progress. Conclusion We have used intraoperative CBCT imaging with automatic registration in the OR with four patients and achieved consistently reliable tracking in each case. Initial results demonstrate that TRE distributions obtained from marker-based registration provide better coverage over a surgical region of interest when compared to anatomical pointbased registration, with further investigations focused on optimal marker geometry. Provided that intraoperative scanning is available, the automatic registration technique is feasible to integrate into the surgical OR. We will continue to investigate this registration technique for surgical navigation in 15 more extremity patients, and future studies involving pelvic sarcoma tumor resection.

Accuracy of hybrid electromagnetic tracking and personalized guides for glenoid models A. W. L. Dickinson1, B. J. Rasquinha2, D. R. Pichora2,3, R. E. Ellis1,2,3 1 Queen’s University, School of Computing, Kingston, Canada 2 Queen’s University, Department of Mechanical and Materials Engineering, Kingston, Canada 3 Queen’s University, Department of Surgery, Kingston, Canada Keywords Personalized guides  Surgical navigation  Electromagnetics  Orthopedic surgery Purpose A shortcoming of personalized surgical guides is that there is seldom an easy way to determine whether the guide properly fits the target

Int J CARS anatomy. If a poor fit is detected intraoperatively, there are few alternative technologies. We propose to unify personalized guides with image-guided surgery (IGS) navigation. A first step is to track a personalized guide, permitting an easy check of accuracy and IGS-navigation conversion. Technologically, optical localization is limited by the line-of-sight requirement and large trackers that are difficult to attach and rigidly affix to small personalized guides. Electromagnetic (EM) tracking has been shown to be intraoperatively feasible [1]; its apparent low accuracy [2] can be overcome with recent calibration algorithms [3, 4] that go beyond conventional registration methods. This work investigated the combination of registration by a personalized guide with the flexibility of IGS navigation. The high inherent orientation accuracy of EM was used to attain sub-millimeter calibration and application accuracy. It improves previous work [5] by registering lines to lines, rather than by using points as data. A pre-clinical trial used the glenoid region of the shoulder as a combined registration feature using the target anatomy. This has proved to be a challenging anatomy for personalized guides [1] so it provides opportunities for improvements. Methods A commercial EM system (Aurora, NDI, CA) was used for experiments. With approval by the relevant Institutional Review Board, CT scans of 10 cadavers were segmented and the scapulas were created by additive manufacturing, as previously described [4]. Preoperative plans included 5 holes for reverse shoulder athroplasty glenoid baseplate alignment and fixation; these were subtracted and resulted in through holes in the physical models. As is commonly done for personalized guides, a registration device was created with a ‘‘negative surface’’ that mated to the target anatomy; here, to the superior glenoid. The device had 8 through holes for calibration and was affixed to an EM disc-style sensor. The EM sensor was calibrated to the negative surface using a paired-lines adaptation of a previous algorithm [3] by means of a sharp EMtracked probe. The adaptation minimizes the sum of squared inter-line distances between the computer model and probed lines, rather than aligning the least-squares crossing points as in [3]. A guide, its ‘‘negative surface’’, and fiducial probing are pictured in Fig. 1. Fiducial localization error was computed from line–line distances and angles.

were collected and averaged. Target registration error was computed from line–line distances and angles.

Fig. 2 (A): A photo of a representative scapula model with reference EM and and personalized guide. (B): A photo of the same model with personalized guide; the guide is outlined. (C): A photo of the same model with the manufacturer stock probe placed in one of the planned ‘‘negative channels’’ Results The tracked guides were characterized with an angular Fiducial Localization Error (FLE) of 0.7 ± 0.4 and a positional FLE line– line distance of 0.3 mm ± 0.2 mm; full tabulated FLE results for each model are given in Table 1. The time needed to perform the equivalent of intraoperative calibration of the tracked guides was 167 s ± 39 s. Table 1 Angular and Positional FLE results for the 10 personalized guides, 8 probed lines in each. Mean (l) and standard deviations (r) can be seen below S01

S02

S03

S04

S05

S06

S07

S08

S09

S10

Pooled

Angular FLE l ()

0.5

0.7

0.8

0.8

0.7

0.7

0.6

0.6

0.6

0.7

0.7

Angular FLE r ()

0.3

0.4

0.4

0.3

0.3

0.4

0.4

0.4

0.4

0.5

0.4

Positional FLE l (mm)

0.2

0.4

0.2

0.2

0.3

0.2

0.2

0.3

0.4

0.3

0.3

Positional FLE r (mm)

0.2

0.2

0.2

0.1

0.1

0.1

0.2

0.3

0.2

0.2

0.2

The Target Registration Error (TRE) was used to evaluate the difference between the actual negative channels in the model scapulas and the tracked position and direction of an EM probe placed in each channel. The angular TRE was 1.6 ± 0.8 and the positional TRE was 0.4 mm ± 0.3 mm; full tabulated TRE results for each model are given in Table 2. Table 2 Angular and Positional TRE results for the 10 scapulae models, 5 probed target lines in each. Mean (l) and standard deviations (r) can be seen below

Fig. 1 (A): A photo of a representative personalized glenoid guide with attached EM sensor. (B): A photo of the personalized guide ‘‘negative surface’’ designed to mate with the glenoid, highlighted and bordered. (C): A photo of the manufacturer’s stock probe being used to characterize the guide A second EM sensor acted as an anatomical reference on the coracoid process. For each model scapula, the ‘‘negative surface’’ of the characterized guide was mated with its corresponding glenoid region and 3 s of EM data were collected, shown in Fig. 2b. As done in a clinical workflow, the guide was then removed. Each of the 5 negative channels in the glenoid was then probed; 3 s of EM data

S01

S02

S03

S04

S05

S06

S07

S08

S09

S10

Pooled

Angular TRE l ()

1.9

0.9

1.7

1.9

1.2

1.3

2.2

2.0

1.6

1.1

1.6

Angular TRE r ()

0.2

0.7

0.8

1.1

1.0

0.5

0.4

1.5

1.7

0.6

0.8

Positional TRE l (mm)

0.4

0.6

0.5

0.2

0.3

0.4

0.4

1.0

0.2

0.3

0.4

Positional TRE r (mm)

0.2

0.5

0.3

0.2

0.3

0.3

0.3

0.7

0.3

0.2

0.3

Using a two-tailed rank-sum test, the angular error was not statistically different from previous work [4] (p = 0.54) and the positional error was statistically significantly better (p \ 0.002). Submillimeter accuracy was achieved.

123

Int J CARS Conclusion We achieved sub-degree and sub-millimeter TRE accuracy in a laboratory application of EM tracking with personalized guides. The guides provided anatomical registration and the EM provided information usable in an IGS system. The reported values are consistent with previously published error and, together, suggest that accurate characterizations were achieved [3, 4]. The FLE numbers are much lower than might be expected from EM tracking; the accuracy results from the registration that accounts strongly for direction. The algorithm is an improvement on Rasquinha’s ‘‘crossing lines’’ algorithm [3], removing the need for geometric intersection. This facilitated a better guide design while maintaining high accuracy. The result is an ergonomic system, with personalized guides for registration in a conventional IGS setting. Results were limited to a laboratory study on the shoulder. The metal retractors used in shoulder arthroplasty may interfere with the EM field; further investigation is warranted. The tracked guides still require preoperative imaging, both segmented and preoperatively planned. However, the intraoperative calibration time suggests that it could be done by a perioperative assistant during surgery, so it would not likely add to intraoperative time. The primary contribution of this work is a method to help mitigate the challenges of reliably registering personalized guides to difficult anatomy. Currently, if a guide fits poorly, there is no way to verify the registration. A tracked guide enables intraoperative verification and, if necessary, conversion to IGS navigation. EM was used for tracking because its devices are small and light, so they can be incorporated into personalized guides with minimal interference. Future work will include cadaveric studies and pilot clinical trials. References [1] Lugez, E., Sadjadi, H., Pichora, D.R., Ellis, R.E., Akl, S.G., Fichtinger, G.: Electromagnetic tracking in surgical and interventional environments: usability study. Int J Comput Assist Radiol Surg 2015; 10(3):253–262 [2] Frantz, D.D., Wiles, A., Leis, S., Kirsch, S.: Accuracy assessment protocols for electromagnetic tracking systems. Physics Med Biol 2003; 48(14), 2241 [3] Rasquinha, B.J., Dickinson, A.W.L., Venne, G., Pichora, D.R., Ellis, R.E.: Crossing-lines registration for direct electromagnetic navigation. Med Image Comput Comput Assist Interv 2015:321–328 [4] Dickinson, A.W.L., Rasquinha, B.J., Rudan, J.F., Ellis, R.E.: Personalized guides for registration in surgical navigation. Stud Health Technol Inform 2016; in press [5] Verborgt O., Vanhees M., Heylen S., Hardy P., Declercq G., Bicknell R.: Computer navigation and patient-specific instrumentation in shoulder arthroplasty. Sports Med Arthrosc Rev. 2014; 22(4):e42-e49

Safety and feasibility study for real-time electromagnetic navigation in breast-conserving surgery G. Gauvin1, T. Ungi1,2, A. Lasso2, C. Yeo1, R. Walker1, J. Rudan1, G. Fichtinger1,2, J. Engel1 1 Queen’s University, Surgery, Kingston, Canada 2 Queen’s University, School of Computing, Kingston, Canada

123

Keywords Breast-conserving surgery  Electromagnetic tracking  Image-guided surgery  Breast cancer Purpose Breast cancer, the most common cancer in women, is ideally treated with breast-conserving surgery in its early stage. Precise delineation of the tumor margins is difficult as lesion margins are commonly not physically palpable, and breast tissue moves and deforms during the procedure [1]. The presence of a positive margin is linked with an increased local recurrence rate despite adjuvant radiotherapy, and therefore leads to the patient requiring additional surgeries and treatment. Current strategies have a reexcision rate for positive margins as high as 25 % [2]. We have developed a real-time electromagnetic (EM) navigation system via ultrasound (US) to register the tumor resection volume from a tracked needle fixed in the tumor, allowing tumor movement to be followed in real-time during surgery [3]. This method has the potential to reduce the incidence of positive margins, while reducing the amount of healthy tissue removed. A previous study done on breast phantoms showed a decrease in positive margin rate from 42.9 % (wire-localization) to 19.0 % (EM navigation). The goal of this prospective phase 1 study was to assess the feasibility of using our electromagnetic navigation system in the operating room. Methods Female patients with a single palpable tumor were recruited to undergo a partial mastectomy. Intraoperatively, the SonixGPS Tablet (Ultrasonix, Vancouver, CA) ultrasound with its built-in EM tracker was used to register the tumor resection volume (Fig. 1). The surgeon inserted a wire-localization needle in the tumour under ultrasound guidance (Fig. 2A) and performed tumour contouring via an intraoperative touchscreen interface (navigation tablet). The SlicerIGT navigation software (www.SlicerIGT.org) provided real-time visualization of the resection volume and cautery position with respect to the tumor. The resulting real-time navigation was displayed on a laparoscopy monitor (Fig. 2B). In this study, feasibility was assessed via three components: confirmation of safety and sterility, measurement of the duration of operation and tumor registration, and completion of a surgeon questionnaire.

Fig. 1 Operating room setup of the breast surgery electromagnetic navigation system. The black coils are electromagnetic position sensors. The navigation tablet allows the surgeon to manually select the tumor resection volume in a sterile fashion. The resulting realtime navigation is also displayed on a laparoscopy monitor

Int J CARS Image-guided navigation surgery for pelvic malignancies using electromagnetic tracking and intra-operative imaging J. Nijkamp1, K. Kuhlmann1, J.- J. Sonke1, T. Ruers1 1 Netherlands cancer institute - Antoni van Leeuwenhoek, Surgery, Amsterdam, Netherlands Keywords Image guided surgery  Navigation  Abdominal surgery  EM tracking

Fig. 2 A. Needle insertion under ultrasound guidance. B. Breastconserving surgery using visual feedback on the display monitor Results Ten patients with a mean age of 58.9 years (range 29–92 years), diagnosed with a stage IA to IIIA breast cancer (n = 8) or a benign breast lesion (n = 2) were recruited. The mean operative time was 62.5 min (range 48–82 min) for the cases of partial mastectomy with sentinel node biopsy (n = 8), and 35.5 min (range 29–42 min) for the cases of partial mastectomy alone (n = 2). Mean registration time was 8.25 min (range 5–12 min). There were no EM-specific complications or breach in sterility during surgery. Feedback questionnaires stated that none of the participants found that the EM sensors interfered with the surgical procedure. Using the 5-point Likert scale, participants stated that it was somewhat easy to complete the registration process (n = 7)(n = 1 very easy; n = 1 neither easy/ difficult; n = 1 somewhat difficult) and to use the electromagnetic system to guide the surgery (n = 7)(n = 2 very easy; n = 1 neither easy/difficult), even without formal prior ultrasound training. Conclusion This study shows that EM navigation is feasible and safe to use intraoperatively in breast-conserving surgery. This technology could provide real-time feedback to surgeons that may improve treatment outcome. These encouraging results support the next phase of research: a trial on non-palpable tumors. References [1] Ananthakrishnan P, Balci FL, Crowe JP. (2012). Optimizing Surgical Margins in Breast Conservation. Int J Surg Oncol, Epub. [2] Krekel NM, Haloua MH, Lopes Cardozo AM, de Wit RH, Bosch AM, de Widt-Levert LM, Muller S, van der Veen H, Bergers E, de Lange de Klerk ES, Meijer S, van den Tol MP. (2013). Intraoperative ultrasound guidance for palpable breast cancer excision (COBALT trial): a multicentre, randomised controlled trial. Lancet Oncol, 14(1):48–54. [3] Ungi T, Gauvin G, Lasso A, Yeo C, Pezeshki P, Vaughan T, Carter K, Rudan J, Engel C, Fichtinger G. Navigated breast tumor excision using electromagnetically tracked ultrasound and surgical instruments. IEEE Trans Biomed Eng (in press).

Purpose Image-guided navigation surgery has been around for over two decades, but application in abdominal surgery is still limited. The main reason is the frequent anatomical changes between pretreatment images and the actual surgery. However, part of the pelvic anatomy such as the iliac arteries and veins, their surrounding lymph nodes, and tumors attached to the pelvic wall, are relatively rigid. Use of a navigation system could improve the anatomical insight derived from preoperative imaging. The purpose of this study was to implement a surgical navigation system for pelvic surgery, and to evaluate the beneficial value of the system in surgery of rigid pelvic malignancies. Methods For tracking, a NDI (Northern Digital Inc, Waterloo, Canada) Aurora V2 electromagnetic (EM) system with a tabletop field generator (work field 42x60x60 cm) was used. In-house developed navigation software acquired OpenIGTLink TRANSFORM ( http://www.igstk.org) sensor positions at 10 Hz using the PlusServer from the Plus Toolkit (https://www.assembla.com/spaces/ plus/wiki). For patient tracking a Philips Traxtal sticker set was used, with three stickers containing each two 5 degree of freedom (DOF) EM-sensors (Fig. 1). During surgery a 6DOF sterile pointer was used. One day before surgery a CT scan was acquired with the stickers placed and marked at the lumbar curvature, and the anterior superior iliac spines. From the CT scan, the EM-sensors, pelvic blood vessels, ureters, and bones were segmented (semi)automatically (Fig. 2). The tumor and lymph nodes were segmented manually after registration with MR and PET imaging. During surgery the patient was positioned on the field generator embedded in a dedicated matrass and the stickers were re-applied. A registration was made between the EM-sensor positions derived from the NDI system, and the positions in the CT scan.

Fig. 1 Patient localization stickers with EM-sensors placed on the right iliac spine (left), the left iliac spine (middle) and in the lumbar curvature (right). The stickers were placed and marked for the CT scan and were re-applied just before surgery

123

Int J CARS the planning CT using chamfer matching. Subsequently, the actual position of the reference stickers was updated, and the new RMS was reported. All patients scheduled for open surgery of pelvic malignancies which were assessed to be rigidly attached to pelvic structures could be included in a prospective, review board approved pilot study. Results Twenty patients were included, 12 primary or recurrent colorectal cases with extra-mesorectal pathologic lymph nodes, 5 pelvic lymph node recurrences or urologic tumors, two recurrent sarcomas, and one gynecological local recurrence. Segmentation of the anatomical models took on average 94 min (range 54–124 min). The RMSE of the external markers registration was on average 0.75 cm (range 0.31–1.58 cm). In 11 cases the patient was setup in French position (legs spread) during surgery, which was different from the scanned setup. The average registration error (RMSE) was 1.2 cm (range 0.3–2.0 cm) for patients in French position, and 0.8 cm (range 0.3–1.2 cm) for patients in a straight setup. In 4 out of the 8 patients operated after the introduction of the registration, manual override, adjustments were made ranging from 0.3 cm to 1.1 cm. After the manual adjustments, localization of the known anatomical structures was within 5 mm in all cases. In the last two patients, reevaluation of the sticker positions with respect to the bony anatomy using intraoperative imaging resulted in a sensor registration error of only 0.1 cm RMS. In all cases where one or both ureters were localized in order to spare them, the navigation system resulted in fast (within 5 min) and accurate (within 4 mm) localization. In all cases the surgeons (n = 10) indicated that the system improved preoperative and intraoperative anatomical insight, and decreased time to localize the tumor and/ or malignant nodes. Iliac, aortic and obturator lymph nodes as small as 4 mm on CT imaging were localized and removed. Conclusion We successfully implemented an image guided surgical navigation system for pelvic surgery. Especially when patients are positioned differently between imaging and surgery, large registration errors occur when using skin marks for patient localization. Manual adaptation of the registration using bony landmarks resulted in improved navigation accuracy. Actual intra-operative imaging to localize reference sensor with respect to bony anatomy improved the accuracy of the system considerably. Our surgeons are enthusiastic and we will further refine the system for future use, also in surgical navigation of non-rigid targets.

Intraoperative user interface for navigated breast tumor surgery

Fig. 2 Example of a navigated external iliac lymph node dissection. Top: localization of the lymph node using the tracked pointer. Note that the lymph node area is closed and the lymph node is not visible. Middle: screenshot of the navigation system at the same time (lymph node in green). Bottom, dissection of the lymph node after opening the cavity At each procedure, navigation accuracy was checked by pointing at the aorta bifurcation, and the left and right common iliac artery bifurcation without viewing the navigation system. Subsequently, the system was used to localize the ureters (if needed for surgery) and the tumor/lymph nodes (Fig. 2). After 10 patients, the navigation software was extended with a manual override of the external marker registration. Based on bony anatomical landmarks which were localized by the surgeon, the registration could be adjusted in left– right, cranio-caudal and anterior-posterior direction. The needed adjustments were evaluated. In the last two procedures, intra-operative imaging (Philips Allura FD20) was used to re-evaluate the sticker positions with respect to the bony anatomy. The intra-operative scan (field of view 25 x 25 x 19 cm) was registered on bony anatomy to

123

T. Ungi1, T. Vaughan1, G. Gauvin1, P. Pezeshki1, A. Lasso1, C. J. Engel 1, J. Rudan1, G. Fichtinger 1 1 Queen’s University, Kingston, Canada Keywords Lumpectomy  Breast cancer  3D slicer  Electromagnetic tracking Purpose Breast conserving surgery is the most common therapy for early stage breast cancer. However, the first procedure often fails to remove the entire tumor. Studies report 15–50 % rate of incomplete excisions, when patients undergo another surgery to extend the margins of the first excision attempt. Breast conserving surgery can be navigated by tracking the surgical cutting device and the tumor through a localization wire [1]. This method was found safe and feasible in patients, and is currently under clinical evaluation for effect on success rate in non-palpable breast tumor cases. The next step in the clinical translation process is to develop a user interface that does not require technical staff in the operating room. Operators need to interact with the software through sterile gloves, they need to define tumor margins using tracked ultrasound, and the 3-dimensional visualization scene needs to be adjusted to the operator‘s point of view. We addressed

Int J CARS these issues by providing an open-source solution that can be reused in similar clinical applications. Methods We implemented a minimal graphical user interface for the 3D Slicer application core functions, with components commonly needed in image-guided medical interventions. Buttons for interactions were optimized for touchscreen tablets. They can be used through sterile bags and surgical gloves. Tumor contouring interactions were designed for holding the ultrasound scanner in one hand and operating the touchscreen with the other hand. While the operator touches the visible tumor margins on the ultrasound image, a 3-dimensional margin model is automatically updated using the position tracker of the ultrasound. Virtual camera orientations for navigation scenes are set up by touchscreen interactions and positioning the surgical cutting device in the line of sight of the surgeon. The intraoperative interface for breast tumor surgery was tested by eight surgical residents on synthetic phantoms (Fig. 1). Their experience was measured on a semi-quantitative 5-grade rating scale from worst (1) to best (5). Conventional keyboard and mouse served as control method in our study. Tracking and imaging hardware devices of the navigation system were handled by the PLUS toolkit [3]. Reusable components of the navigation system were implemented in the SlicerIGT extension (www.slicerigt.org) of the 3D Slicer application platform. 3D Slicer (www.slicer.org) offers a convenient plug-in mechanism with an app store [2].

touchscreen is currently under clinical testing in breast cancer surgery (Fig. 2).

Fig. 2 Intraoperative touchscreen user interface during tumor contouring (left) and excision navigation (right) References [1] Ungi T, Gauvin G, Lasso A, Yeo C, Pezeshki P, Vaughan T, Carter K, Rudan J, Engel C, Fichtinger G. Navigated breast tumor excision using electromagnetically tracked ultrasound and surgical instruments. IEEE Trans Biomed Eng (in press). [2] Fedorov A, Beichel R, Kalpathy-Cramer J, Finet J, FillionRobin JC, Pujol S, Bauer C, Jennings D, Fennessy F, Sonka M, Buatti J, Aylward S, Miller JV, Pieper S, Kikinis R. 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magn Reson Imaging. 2012 Nov;30(9):1323–41. [3] Lasso A, Heffter T, Rankin A, Pinter C, Ungi T, Fichtinger G. PLUS: open-source toolkit for ultrasound-guided intervention systems. IEEE Trans Biomed Eng. 2014 Oct;61(10):2527–37.

Topologically consistent triangulation for computer assisted surgery planning 1

M. S. Hefny1, J. J. Peoples1, M. Zec2, D. R. Pichora2, R. E. Ellis1,2 Queen’s University, School of Computing, Kingston, Canada 2 Queen’s University, Department of Surgery, Kingston, Canada 1

Fig. 1 Experiment for testing of the intraoperative user interface on plastic phantoms Results The surgical navigation software for breast cancer was developed in the Python programming language as an extension of 3D Slicer. It is available in 3D Slicer, and the source code is available in a public repository (https://github.com/SlicerIGT/LumpNav) with a license that allows modifications, commercial and academic use without restrictions. Hardware devices for sterile operation technique have been developed and shared as editable model files in the PLUS toolkit (www.plustoolkit.org). These models can be replicated with a 3D printer. Users found the navigation software to be more conveniently usable with the intraoperative interface compared to keyboard and mouse interface: 5 (4–5) versus 2 (2–2.5), p Conclusion Navigated breast tumor surgery is feasible using an intraoperative touchscreen user interface without additional technical staff. The presented interface was rated significantly better than conventional keyboard and mouse. The navigation software with intraoperative

Keywords Scaphoid fixation  Automatic planning  Shape Atlas  Lie groups Purpose Accurate shape atlases are a useful automatic planning strategy for computer-assisted surgery [1]. A recent trend is to use Lie groups for atlas construction [2,3,4]. For the purpose of Lie group shape modeling, each shape is described as a set of transformations on the triangles of the base shape, and these transformations are treated as elements of a Lie group. As a result, if all triangles in a mesh are included, vertices shared by adjacent triangles in the base shape are transformed multiple times in the description of altered shapes. This is not only redundant, but there is no guarantee that in using the model, each transformation will result in the position of shared vertices matching between adjacent triangles. Therefore, steps must be taken to ensure topological consistency in the output of Lie group based shape models. One approach is to select an explicit subset of the surface, defined as a subset of the triangles such that every vertex in the mesh is contained in exactly one triangle. By performing transformations on only this explicit subset, each vertex is transformed only once, ensuring topological consistency. However, manually selecting such a

123

Int J CARS

subset of a large mesh is time consuming, and in some cases may not even be possible. In order to make the determination of such an explicit subset automatic, this work proposes a new surface extraction algorithm that will determine both a mesh and an explicit subset simultaneously. This algorithm leverages the tomographic nature of medical images, constructing the surface by extracting contours from each slice of the segmented image, then tiling between contours from each slice. Methods CT scans of scaphoids were obtained from surgical patients. Beginning with the segmented image, contours were extracted from each axial slice using the marching squares algorithm. To create a closed surface from a series of connected contours, there are two things that must be achieved. First, adjacent contours must be connected by a continuous surface, and second, the final contours along the slice axis must be closed. In our case, we add the requirement of both the existence of, and specification for, an explicit subset of triangles in the surface mesh. Tiling between contours is rigorously developed [5], and we therefore needed only to modify the algorithm to ensure the existence of an explicit subset. However, we do not need the explicit subset between every pair of contours. In fact, since every explicit subset between such a band contains every vertex of the two contours exactly once, the explicit subsets of any two adjacent bands would include the vertices of the central contour twice. As such, starting from the first band, we can take explicit subsets from every other band in order to form an explicit subset for the entire surface. If the number of bands is odd, then this will cover both the first and last bands, and thus every vertex will be accounted for. Otherwise, the last band will be an implicit band, and therefore the last contour will not be contained in the explicit subset. This situation is handled by the closing of the surfaces. Closing the final contours is simple: if the band is explicit, the vertices are already accounted for, and we can simply use any polygon triangulation to close the shape. If the band is implicit, we must account for the vertices of the final contour. For the purposes of this study, it was sufficient to duplicate and inset the final contour, creating an additional explicit band while maintaining the original geometry, and then close the new contour as in the explicit case. The consistently triangulated meshes were encoded as transformation matrices of the Special Euclidean group (SE(3)), and a matrix Lie group atlas was constructed [4]. A surgical plan was derived for the base shape using manual plans of three expert surgeons. The derived plan was carried on the base shape, registered to 18 scaphoid samples, and compared to actual plans devised by three expert surgeons. Results One interesting comparison criterion is the parsimony of the atlas. For dimensionality reduction, it is desirable to minimize the number of deduced descriptive features required to cover most of the variance in the population. The first component of standard PCA covered 53 % of the population variance, with 6 components needed to cover 95 % of the variance. The first component of the SE (3) atlas covered 86 % and only 3 components were needed to cover 95 %. It is clear that the SE (3) implementation was more parsimonious than the standard implementation of PCA. Fig. 1 shows the plots of the accumulated eigenvalues of both methods.

123

Fig. 1 The Parsimony Spectrum for both standard PCA and SE(3) encoding To verify the automatic planning, the 18 non-base samples were planned by three expert surgeons using custom software. For each sample, the atlas was then constructed leaving the target sample out. The base shape, carrying the plan, was fitted to the target sample using the atlas. The computed plan was compared to each of the three plans drawn by the surgeons in terms of the minimum crossing distances and the angular differences between the two axes. The derived plans differed from the surgeons’ plans less than surgeons differed from each other. Using minimum-crossing distance, the atlas-surgeon difference had a mean of 0.8 mm ± 0.5 mm; the surgeon–surgeon difference had a mean of 1.7 mm ± 1.4 mm. Using line angles, the atlas-surgeon difference had a mean of 3.9 ± 2.5; the surgeon–surgeon difference had a mean of 5.7 ± 3.3. Table 1 summarizes the results. Table 1 Surgeon-Surgeon Comparison vs. Atlas-Surgeon Comparison Surgeon– Surgeon

AtlasSurgeon

Minimum crossing distances (mm) 0.80

0.47

Angular differences (degrees)

3.31

5.67

Conclusion The topologically consistent meshes produced an accurate and precise atlas for automatic planning of computer-assisted surgeries. The method can be extended to other orthopedic and general surgeries. References [1] Hefny MS, Rudan JF, Ellis RE (2015) Computer assisted hip resurfacing planning using Lie group shape models. International Journal of Computer Assisted Radiology and Surgery 10(6):707–715.

Int J CARS

[2]

[3]

[4]

[5]

Hefny MS, Rudan JF, Ellis RE (2014) A matrix Lie group approach to statistical shape analysis of bones. Studies in Health Technology and Informatics 96:63–169 Hefny MS, Pichora DR, Rudan JF, Ellis RE (2014) Manifold statistical shape analysis of the distal radius. International Journal of Computer Assisted Radiology and Surgery, 9(SUPP 1): S35–S42. Hefny MS, Okada T, Hori M, Sato Y, Ellis RE (2015) A liver atlas using the special Euclidean group. Frangi et al. (Eds.): MICCAI 2015, Part II, LNCS 9350: 238–245. Fuchs H, Kedem ZM, Uselton SP (1977) Optimal surface reconstruction from planar contours. Communications of the ACM 20(10): 693–702.

ArthroPlanner: a surgical planning solution for acromioplasty C. Charbonnier1, S. Chague´1, B. Kevelham1, F. C. Kolo2, A. La¨dermann3 1 Artanim Foundation, Medical Research Department, Meyrin, Switzerland 2 Rive Droite Radiology Center, Geneva, Switzerland 3 La Tour Hospital, Division of Orthopedics and Trauma Surgery, Meyrin, Switzerland Keywords Acromioplasty  Preoperative planning  Simulation  Shoulder Purpose Subacromial impingement of the rotator cuff between the anterior or lateral acromion and the superior humeral head is a common disorder. This condition arises when the subacromial space height is too narrow during active elevation or scaption of the arm above shoulder level due to an abnormal hooked shape or large lateral extension of the acromion. In severe cases of impingement syndrome, an arthroscopic acromioplasty surgery is usually performed to resect the different areas of the acromion causing damage to the subacromial structures. The exact location and the amount of bone to be resected is generally left to the unique appreciation of the orthopedic surgeon during surgery. To improve the precision of this resection, surgeons could greatly benefit from a surgical planning solution that aims at providing precise information about the surgical procedure. Moreover, since subacromial impingements are the result of a dynamic mechanism, an effective planning solution should analyze both the morphological joint’s structures and its dynamic behavior during shoulder movements to fully apprehend the patient joint’s condition. We present our computer-assisted planning solution ‘‘ArthroPlanner’’ for acromioplasty. The solution allows to perform standard morphological bony measurements, as well as 3D simulations of the patient’s joint during everyday shoulder activities. The software computes the precise bone resection (location and amount) based on detected subacromial impingements during motion. Methods We reconstruct the bones of the patient’s shoulder joint (scapula and humerus) from a CT image using Mimics software. The bones are then imported into ArthroPlanner software and the following steps are performed: First, biomechanical parameters are computed. The glenohumeral joint center is calculated by a sphere fitting technique [1] (Fig. 1A). Bone coordinate systems are established for the scapula and humerus (Fig. 1B) based on ISB [2] using anatomical landmarks defined on the bone models and CT image.

Fig. 1 A) Glenohumeral center computation, B) bone coordinates systems computation and C) acromial resection plan Second, motion is applied at each time step to the humerus model with real-time evaluation of impingement. The minimum humeroacromial distance [3] is calculated based on the simulated bones models positions. A color scale is used to map the variations of distance on the scapula surface (red color = minimum distance, other colors = areas of increased distance). Given the thickness of the potential impinged tissues, subacromial impingement is considered when the computed humero-acromial distance is \ 6 mm [3]. To test a wide variability of realistic movements, a motion database of daily activities (e.g., cross arm, comb hair) is used in addition to standard kinematic sequences (e.g., elevation, scaption). Third, the acromial resection plan is defined based on the 3D simulation results. A color map is used to represent areas where impingements occurred between the acromion and humerus (Fig. 1C). The red color denotes the area with the smallest humero-acromial distance computed over the different motion simulations. A PDF report is finally generated that contains patient’s information and the measurements performed. The bones and the simulation data are also exported to be used in a simple 3D viewer dedicated to the surgeon. With this viewer, the surgeon is able to play all simulations, observe impingements dynamically and review the resection plan. Results To test the validity of the planning solution, a clinical study is performed with 67 patients undergoing acromioplasty by an experienced orthopedic surgeon. 3D reconstruction and preoperative planning were performed for all patients. However, the surgeon could review the results of the planning for 32 patients only, the other 35 were part of the control group. The software showed significant robustness in performing the different planning steps and provided intra-patient reproducible results. One planning with the 3D reconstruction took in average 45 min, which is feasible in the clinical routine. The surgeon reported that the planning changed completely his way to handle the surgery. He has decreased the number of anterior acromioplasties, and is performing more lateral and posterior bony resections. Post-operative visits for all patients will be performed at 6 months and 1 year after surgery, including a clinical examination (evaluation of the ranges of motion, pain scores) and an echography to control the rotator cuff. A post-operative CT 3D reconstruction will also be performed to determine the actual bone resection executed at surgery compared to the planning recommendations. The data collected will be compared between the groups. Conclusion We presented a computer-assisted solution for acromioplasty. The software allows surgeons to better plan the surgical procedure by visualizing dynamic simulation of the patient’s shoulder joint during everyday activities. Impingements are dynamically detected and the exact location and amount of bone to be resected is precisely computed. As a result, the success of the acromioplasty does not only rely on the surgeon’s experience, but on quantitative data. Although the clinical validation of the planning solution is currently under evaluation, we expect that it will allow patients to

123

Int J CARS recover more effectively postoperative joint mobility, get a better relationship with pain and a better healing rate of the rotator cuff tendons. References [1] Schneider P, Eberly D. Geometric tools for computer graphics. The Morgan Kaufmann Series in Computer Graphics and Geometric Modeling, 2003. [2] Wu G, van der Helm F, Veegerc H, et al. ISB recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion - Part II: shoulder, elbow, wrist and hand. J Biomech, 38(4):981–992, 2005. [3] Charbonnier C, Chague´ S, Kolo FC, La¨dermann A. Shoulder Motion During Tennis Serve: Dynamic and Radiological Evaluation Based on Motion Capture and Magnetic Resonance Imaging. Int J CARS, 10(8):1289–1297, 2015.

A low-cost system for pericardiocentesis training

image-guided

Five phantoms can be prepared under an hour without taking into account the time needed for the gelatin to set in the refrigerator. The Sonix Touch cart ultrasound machine (Analogic Corp., Peabody, MA, USA) with the 3D Guidance trakSTAR electromagnetic tracking system (Northern Digital Inc., ON, Canada) were used while testing the phantom. A reference sensor was placed underneath the phantom and a tracked needle was used to drain the effusion (Fig. 1). The electromagnetic tracker was connected to a computer running the PLUS software toolkit’s PlusServer application ( https://www.assembla.com/spaces/plus/wiki). PlusServer relayed all tracking and image data to 3D Slicer, a commonly used open-source platform for image-guided interventions (http://www.slicer.org/). Using basic features of SlicerIGT (http://www.slicerigt.org/wp/), an extension for 3D Slicer, models representing the needle tip were created and visualized relative to the ultrasound image plane. The tip of the needle was marked with a sphere, allowing users to distinguish between the tip of the needle and the needle shaft (Fig. 2).

computer-navigated

V. Harish1, A. Baksh1, T. Ungi1, R. Pal2, G. Fichtinger1 1 Laboratory for Percutaneous Surgery, School of Computing, Queen’s University, Kingston, Canada 2 Department of Cardiology, School of Medicine, Queen’s University, Kingston, Canada Keywords Medical training  Phantom  Ultrasound  Pericardiocentesis Purpose Canadian medical universities are in the process of transitioning to a Competency-Based Medical Education (CBME) model at all levels of training. Achievement of competence requires practice, and for reasons of patient safety and trainee comfort, the early stages of practice are best done in a simulated environment [1]. Phantoms simulating human anatomy have been created to objectively evaluate trainees in a variety of medical procedures. Pericardiocentesis is a relatively rare, but high-risk procedure that involves the aspiration of fluid from the pericardial cavity to relieve compression of the heart. The current standard of care involves the use of ultrasound imaging to provide real-time visualization of the needletip in relation to surrounding organs [2]. Commercially available pericardiocentesis phantoms, such as Blue Phantom’s Transthoracic Echocardiography and Pericardiocentesis Ultrasound Training model (CAE Healthcare Canada, QC, Canada), are extremely expensive ( http://www.bluephantom.com). Inexpensive phantoms have common limitations such as the lack of realistic anatomical landmarks and the inability to pump [3, 4]. Thus, our goal was to create a low-cost, realistic phantom that could be used alongside an open-source software platform for image-guided pericardiocentesis intervention training. Methods Our pericardiocentesis phantom consists of a heart model within a plastic container filled with gelatin. To construct the heart model, one balloon is placed inside a second balloon with pneumatic tubing running out of the inner balloon. This tubing is connected to a 60 cc syringe used to simulate pumping. Each balloon is filled with water of a different color, allowing users to easily determine if they are draining the pericardial effusion or have punctured the heart. The heart model was placed in a container filled with gelatin with the pneumatic tubing running out of the top of the container. Plastic cutting board was cut to look like the sternum and ribs and was placed on top of the gelatin. Silicon skin was placed on top of the ribs. The cost for the first phantom is roughly $10, however as the container, tubing, cutting board, and silicon skin are reusable, the costs decrease further as more phantoms are built.

123

Fig. 1 Schematic diagram (left) and photograph (right) for performing ultrasound image-guided, pericardiocentesis on low-cost phantoms

Fig. 2 Ultrasound images of a phantom with a filled (left) and unfilled inner balloon showing use of SlicerIGT to visualize the needle during insertion (right) Results Our system allows for trainees to practice ultrasound-navigated pericardiocentesis at very low costs. Our phantoms appear sonographically and anatomically realistic. The syringe can be pumped to create a beating effect at 30 beats per minute. Exploring a different pumping mechanism as opposed to manually pumping a syringe may allow us to have a more rapid beating effect. The difference between the filled and unfilled inner balloon is clearly visible under ultrasound, as once the inner balloon is filled it appears hyperechogenic to the water in the outer balloon representing the pericardial effusion (Fig. 2). The two balloon layers appear similar to the pericardium and myocardium, decompression of the myocardium is clearly visible in the ultrasound, and the synthetic ribs and skin provide realistic anatomical landmarks.

Int J CARS The amount of water in the inner and outer balloon can also be changed easily, allowing us to create heart models of various sizes— representing males and females with different sized effusions. Visualizing the tip of the needle aids during the training process since it is easy to mistake the shaft of the needle for the needletip, causing trainees to drive the needle too far and puncture the heart. Our phantom, if refrigerated, can last about a week; however, they are single-use. Five phantoms were created, appearing identical except for the amount of water placed in the outer balloon to simulate different difficulties for the procedure. Needle insertion trials were attempted on each of these phantoms by unskilled, non-clinical students. Three out of the five trials were unsuccessful due to inexperience of the students, however there was no mechanical failure of the phantoms. Conclusion We have created a phantom for pericardiocentesis training using inexpensive materials and demonstrated it with an open-source navigation software in ultrasound-guided pericardiocentesis interventions. Owing to its low cost and manufacturing complexity, the phantom is easily reproducible; and in conjunction with an open source navigation system, it promises to be a viable tool in training residents to perform pericardiocentesis before they are expected to perform it on patients. Future work will involve user-performance studies and adding self-sealing capability to the phantom, allowing it to be used multiple times. References [1] Wang EE, et al. (2008) Developing Technical Expertise in Emergency Medicine—The Role of Simulation in Procedural Skill Acquisition. Academic Emergency Medicine, 15(11), 1046–1057. [2] Loukas M, et al. (2012) Pericardiocentesis: A clinical anatomy review. Clinical Anatomy, 25(7), 872–881. [3] Campo Dell’orto M, et al. (2013) Assessment of a Low-Cost Ultrasound Pericardiocentesis Model. Emergency Medicine International, 2013, n.p. [4] Zerth M, et al. (2012 Dec) An Inexpensive, Easily Constructed, Reusable Task Trainer for Simulating Ultrasound-Guided Pericardiocentesis. The Journal of Emergency Medicine, 43(6), 1066–1069.

Ontology-based instruments detection framework for surgical assistance

in

a context-aware

H. Nakawala, G. Ferrigno, E. De Momi Politecnico di Milano, Department of Electronics, Information and Bioengineering (DEIB), Milan, Italy Keywords Context-aware surgery  Ontology-based configuration  3D object segmentation  Surgical assistance Purpose Knowledge-driven context-aware framework [1] interprets ontological surgical process models and provides assistance by providing decision support, e.g. contextual information, to the surgeon at a specific instance of a surgical activity, which could help to reduce procedure-related complications. To assist surgeons during the interventions (for example, to suggest the best available surgical instrument fitting the current step) accurate and consistent detection of instruments located on the surgical stand using vision-based sensors, can be indeed a useful approach. Data-driven approaches to object segmentation provide non-consistent segmentation due to different object features e.g. colour thresholds whose values have to be

manually adjusted in the image processing algorithms for optimal results. The ontology-based configuration of such features might provide consistent results, which could integrate into the developed framework. Methods We have contextualised Thoracentesis, a procedure used to withdraw fluid from the chest. In order to obtain contextual information, we used image-processing algorithms for detection of instruments (as an example, a 50 ml syringe which is used to withdraw the fluid) [1]. Ontology for Thoracentesis was constructed using a top-down approach where information about Thoracentesis was obtained from a journal article and was analysed using the methodology described in [2] and an opinion from the physician. After identifying appropriate classes, the procedure was formalised using an approach similar to [3], where logical sentences were divided into triplets in the format of Phase (Instrument, Step, Body Structure). For example, withdrawal of a large syringe (50 ml) from the intercostal space is expressed as ‘‘Closure (LargeSyringe, WithdrawLargeSyringe, AreaOfInsertionIntercostal)’’. Thoracentesis instruments instances have been created in the ontology whose names are specified same as the name of the surface patches saved in the file system database. Each of these instrument’s instances is linked with phases and steps of Thoracentesis using indicative prepositions (e.g. closure phase has instrument aspiration syringe). After analysing the requirements of an application-specific ontology, we have created classes for image processing algorithms such as ‘‘PlaneSegmentation’’, ‘‘RegionGrowingSegmentation’’ and so on. Each of these segmentation algorithms has a different set of parameters. During the ‘‘Pre-configuration’’ stage, we have manually segmented surgical instruments to obtain optimal configuration parameters for each instrument and verified the segmentation through the visual inspection. After doing five elaborations for each instrument in different positions, we have extracted the mean value of the parameters required for efficient segmentation. Subsequently, ontology has been updated with these parameters defined for each instruments instance and obtained parameters values are specified as a data-type property assertion. In order to segment the instruments, several pre-processing steps have been performed on the raw point-cloud, obtained by Microsoft Kinect, which includes geometric information (for example, points position i.e. XYZ). To downsample the point-cloud and to approximate the region of interest, voxel-grid and passthrough filters have been implemented respectively, which remove the outlier points such as walls [4]. After that, RANSAC-based plane segmentation [5] has been implemented to segment surgical instruments from the plane e.g. a surgical stand. We have tried to segment a 50 ml syringe and a surgical-swab in four different poses in two different illumination conditions e.g. in the cold white fluorescent tube-light and the warm yellow incandescent lamp. Furthermore, procedure ontology was queried at a particular surgical step, decided by the surgeon, to detect surgical instrument during Thoracentesis. Results Figures 1 and 2 shows a comparison of instruments, a 50 ml syringe and a surgical-swab, segmentation results under manual configuration/adjustment and ontology-based configuration of algorithm’s parameters. The segmentation was assessed based on the number of points representing instrument’s surface patches. We were able to detect the surgical-swab more efficiently in the yellow illumination. On the contrary, the syringe was segmented better in the white illumination. However, the approach comes with a limitation when instruments have specular reflections. In our experiments, the developed system was faster than manually adjusting the algorithm parameters, which improves system’s usability.

123

Int J CARS [3]

[4]

[5]

Neumuth T, Strauß G, Meixensberger J, Lemke HU, Burgert O (2006) Acquisition of process descriptions from surgical interventions, In DEXA LNCS, Springer, Bressan S., Kung J., Wagner R. (Heidelberg, Germany), 4080: 602–611. Gupta M, Sukhatme G (2012) Using manipulation primitives for brick sorting in clutter, Robotics and Automation (ICRA), IEEE International Conference On (Saint Paul, Minnesota, USA), 3883–3889. Fischler AM, Bolles CR (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography, Communications of the ACM, 24(6): 381–395.

Intuitive workflow editor for OWL based semantic networks in medical environment Fig. 1 Segmentation of instruments in different positions under white illumination

S. Du¨rr1, A. Huck1, L. Schreiter1, T. Beyl1, J. Giehl2, M. Schwarz2, J. Raczkowsky1, H. Wo¨rn1 1 KIT, Karlsruhe, Germany 2 University Medical Centre, Medical Faculty Mannheim, Heidelberg University, Orthopaedic and Trauma Surgery Centre, Mannheim, Germany Keywords Medical treatment processes  Intuitive workflow editor  Semantic network  Workflow-management systems

Fig. 2 Segmentation of instruments in different positions under yellow illumination Conclusion The ontology-based configuration of algorithm parameters provides consistent segmentation of instruments, comparing the number of points within experiments in each pose, than manual configuration of the algorithm parameters. The ontology-based configuration also enables retrieval of context specific information and processes only required configuration parameters e.g. configuration parameters required to segment 50 ml syringe during withdrawal of the syringe from the chest cavity. However, the developed system experimented with one instrument only which can be further extended to multiple instruments segmentation. References [1] Nakawala H, De Momi E, Morelli A, Tomasina C, Ferrigno G (2015) Ontology-based surgical assistance system for instruments recognition, In Proceedings of CRAS: Joint Workshop on New Technologies for Computer/Robot Assisted Surgery. (In press) [2] Natalya NF, McGuinness DL (2001) Ontology development 101: A guide to creating your first ontology, Stanford knowledge systems laboratory technical reports KSL-01-05 and Stanford Medical Informatics technical reports SMI-2001-0880.

123

Purpose The creation of workflows in medical environment is a time-intensive step and requires medical as well as computer-technical expertise. The existing workflow editors appeared to be too complex and include more functionality than needed. Nevertheless in modern times, the use of computer assistive and system navigated operations in this sector is indispensable. For this reason there is a need for an intuitive editor which offers non-experts the possibility to create valid workflows without technical knowledge of Workflow-Management-Systems. To implement such an intuitive solution for a Workflow-Management-System the purpose is to develop an editor which is user-friendly, has a good performance and encapsulates unused complexity. The focus is on the implementation of an editor which should make it possible for the user to create surgery processes in a simple and smart manner. The overall goal of the project is to develop an intuitive workflow editor based on the YAWL editor [1] which supports the surgery by showing suggested tasks as subsequence using the knowledge of the current selected task. The editor has to extract the logic of possible task sequences out of a database to create a medical treatment process. Furthermore the user should have the opportunity to upload a workflow into the YAWL engine, download a workflow out of the engine and it should allow the user to set local variables as well as various properties. Methods The requirements to a software developement of an intuitive workflow editor are very versatilely. The overall goal is to implement an easy-to-use editor without unnecessary functionality for the definition of the workflows. Our approach is based on the YAWL technology and is inspired by the YAWL editor but does not provide unused features. The process modelling language YAWL is used for developing and illustration of workflows. On the one hand YAWL provides the connection to an engine and on the other hand the technology can represent processes based on the fundament of Petri nets in the YAWL editor as open source software, presented in [1]. Medical treatment processes often have relations and dependencies which are stored in a database. Our implementation uses an OWL database to store data and to represent ontologies. OWL is based on RDF-Syntax and has the advantage to store the data as well as the relation between data. To extract needed data the editor has to be able to establish

Int J CARS SPARQL queries [2]. These queries extract the available tasks (Tasks) out of a specific ontology. It is necessary that the editor is able to read the connections (Transitions) between the extracted tasks and establish the link between them. SPARQL is a language for querying and modifying data sources in RDF format. RDF provides a way to formalize logical statements concerning any entities. SPARQL is often used in a semantic web environment and is based on the syntax of the general SQL Query language. To create medical treatment processes every workflow starts with an input condition continues with valid suggested tasks as well as transitions to connect them and ends with the output condition. Results Our implementation picks up the benefits of the YAWL editor and encapsulates unessential functionality to set up an intuitive workflow editor. For describing dependencies of workflow items we choose the already in-use technologies OWL and YAWL. These were evaluated as the best to be used in the OP:Sense [3] project. The huge amount of flexibility provided is mostly not used by the project but introducing massive complexity to the creation of workflows based on the provided rules. For the evaluation we created conceptual wireframes to communicate our vision to the stakeholders and refine these with the help of their feedback. Using provided OWL ontologies paired with our implementation nearly everybody is able to create given workflows correctly. Suggested tasks in case of branches make it easy and intuitive for the user to choose the right decision by knowing the upcoming task subsequence. One of the problems appeared during the evaluation was the nearly infinite accuracy concerning the description of tasks. For this purpose some kind of drill down opportunities to navigate between different hierarchy levels are needed. As a result we discovered different roles involved in the process of creating such workflows. On the one side there is the role of the specialist and on the other side the workflow creator. The specialist is a person who could be the expert in some section or some type of surgeon. The specialist primary contributes to the semantic net with his technical knowledge and experience about the process. The workflow creator identifies the individual needs of the patient and creates the right workflow path using the benefit of the provided software. It should be clear that these roles can not be clearly separated. For example the newly gained experiences should find their way back into the semantic net. Conclusion We have developed a new editor to generate workflows for medical treatment processes and to avoid unused complexity. Our implementation of an intuitive workflow editor is based on the YAWL technology especially inspired by the YAWL editor. The provided software supports the surgeon by showing suggested tasks as subsequence using the knowledge of the current chosen task. The editor extracts the logic of possible task sequences out of an OWL database. That feature offers non-experts the possibility to create valid workflows. Furthermore the tool allows the user to set local variables as well as various properties. On the one hand these properties are used for the flow control and one the other hand to connect to the YAWL engine. The resulting software is extracted as a standalone out of an eclipse project and can directly be started as an application without using a development environment. Our overall goal to implement an intuitive editor has been reached. Users without technical knowledge of Workflow-Management-Systems are able to set up complex real medical treatment processes in a similar quality to ones designed by experts. The work performed was funded by the Federal Ministry of Education and Research within the project’Konsens OP’. References [1] van der Aalst W, Hofstede AHMT (2003) Yawl: Yet another workflow language, Information Systems, vol. 30, pp. 245–275 [2] Quilitz B, Leser U (2008) Querying distributed rdf data sources with sparql in The Semantic Web: Research and Applications, S. Bechhofer, M. Hauswirth, J. Hoffmann, and M. Koubarakis,

[3]

eds.), vol. 5021 of Lecture Notes in Computer Science, pp. 524–538, Springer Berlin Heidelberg Nicolai P, Brennecke T, Kunze M, Schreiter L, Beyl T, Zhang Y, Mintenbeck J, Raczkowsky J, Wo¨rn H (2013) The OP:Sense surgical robotics platform: first feasibility studies and current research, International Journal of Computer Assisted Radiology and Surgery

Situation detection for an interactive assistance in surgical interventions based on random forests L. Schreiter1, P. Philipp1, J. Giehl2, Y. Fischer3, J. Raczkowsky1, M. Schwarz2, J. Beyerer1,3, H. Wo¨rn1 1 KIT, Karlsruhe, Germany 2 University Medical Centre, Medical Faculty Mannheim, Orthopaedic and Trauma Surgery Centre, Mannheim, Germany 3 Fraunhofer Institute of Optronics, System Technologies and Image Exploitation, Karlsruhe, Germany Keywords Surgical/interventional management

workflow

modelling

&

Purpose In contemporary medicine, the use of assistance functions for diagnosis and surgical interventions is an evolving area [1]. These functions can help to master medical challenges like the prevention of treatment errors, enhancement of outcome and the preservation of a high level of satisfaction for employees as well as patients. To enable such assistance functions in a surgical intervention, we propose a situation detection based on Random Forests. More precisely, the progress of an intervention is deduced by detecting single surgical steps of a pre-modeled workflow. We are convinced, that among other things—e.g. the status of the operating team—this information is a keystone to carry out a tailored assistance function. Methods We have chosen supervised learning [2] for training the models that are used for the detection of the actual progress of a surgical intervention. The idea of supervised learning is to build models which are able, after a learning phase, to deliver correct target vectors {t_1, t_2, t_3…t_n} for new, previously unseen, input vectors {x_1, x_2, x_3…x_n}. To do so the learning phase needs different input and corresponding target vectors which a significant for the identification of steps inside the workflow. 12 datasets of a simplified workflow with 7 surgical steps were recorded and labeled manually. The re-enacted surgical steps differ in the use of tracked instruments, number and position of persons. In Fig. 1 a detailed characteristic of each surgical step is presented. For the implementation of the probabilistic models we decided to use Random Forests as presented in [3] and Support Vector Machine to evaluate the results. For the recording and a future online identification we have choose the OP:Sense Setup [4] and the corresponding perception system [5]. The perception system partly comprised of four Kinect V1 whereby it is possible to recognize people as well as objects in operation theatre. The algorithms outcome delivers the current position of each person, it’s trace on the floor, as well as a representation as a point cloud and skeleton tracking from multiple viewpoints. In the presented approach especially the skeleton tracking deploys data of the characteristic positing of each person in the operation theatre and the total amount of persons. For the identification of the instruments we used ART, a marker based tracking system.

123

Int J CARS

Fig. 1 Specification of the workflow. Subfigure (a) shows the characteristic of the surgical steps a-g. Subfigure (b) depicts the positioning of the operating team in a so called ‘normal positioning’. This position is taken up by the team members in steps a-c and f-g. During the switching, person 1 (P1) and person 2 (P2) change their positing. C1 to C4 are representing the camera positions on the ceiling. IT is the instrument table where the different instruments are placed at the beginning Results To evaluate the performance of the method we used cross-validation with a leave-one-out iterator on the 12 datasets. Thereby each dataset is used once as a test-set while the remaining dataset are for the training set. In Fig. 2 the corresponding normalized confusion matrix are presented. The diagonal elements representing the amount for which the predicted label is equal to the true label, while off-diagonal elements are mislabelled by the classifier. It can be seen the random forest classifier identify most of states.

Fig. 2 Confusion Matrix based on cross validation and leave one out

123

Conclusion In this work we introduced a situation detection to enable an interactive assistance during a surgical intervention. We are convinced, that the current progress of an intervention is essential for providing a tailored assistance function. Therefore, we trained Random Forests to detect 7 different surgical steps of a re-enacted intervention. Both classifier mixed up often the states a and g. One of the reasons for this can be found in the specification of the both states which are indeed similar. In comparison to the Random Forest classifier the SVM identified more often false state as true, e.g. state d, e and f. It can be summarized that the results seems to be promising—Random Forests performed well in the given classification task. For the future, we plan to take our approach to the next level, by combining the classification results with indicators of the operating team status. This will be the starting point for a targeted assistance function. The work performed was funded by the Federal Ministry of Education and Research within the project’Konsens OP’. References [1] Philipp P, Fischer Y, Hempel D, Beyerer J (2016) Framework for an interactive assistance in diagnostic processes based on probabilistic modeling of clinical practice guidelines, in Emerging Trends in Applications and Infrastructures for Computational Biology, Bioinformatics, and Systems Biology (In Press), Elsevier [2] Bishop CM (2006) Pattern Recognition and Machine Learning. Springer [3] Liaw A, Wiener M (2002) Classification, regression by randomforest, R news, vol. 2, no. 3, pp. 18–22 [4] Bihlmaier A, Beyl T, Nicolai P, Kunze M, Mintenbeck J, Schreiter L, Brennecke T, Hutzl J, Raczkowsky J, Wo¨rn H (2015) ROS-based Cognitive Surgical Robotics, Robot Operating System (ROS)—The Complete Reference, pp. 1095–1106 [5] Beyl T, Nicolai P, Comparetti MD, Raczkowsky J, Momi ED, Wo¨rn H (2015) Time-offlight- assisted Kinect camera-based people detection for intuitive human robot cooperation in the surgical operating room, International Journal of Computer Assisted Radiology and Surgery, pp. 1–17

Int J CARS

18th International Workshop on Computer-Aided Diagnosis Chairman: Hiroyuki Yoshida, PhD (USA)

123

Int J CARS Comparative evaluation of deep convolutional neural networks in the improvement of the performance of CAD of polyps in CT colonography A. Oka1, J. Na¨ppi2, T. Hironaka2, D. Regge3, H. Kawahira1, H. Yoshida2 1 Chiba University, Center for Frontier Medical Engineering, Chiba, Japan 2 Massachusetts General Hospital and Harvard Medical School, Boston, United States 3 Institute for Cancer Research and Treatment, Turin, Italy Keywords Computer-aided detection  CT colonography  Neural network  Colorectal cancer Purpose CT colonography (CTC) has been endorsed for colorectal cancer screening by the American Cancer Society, U.S. Multi-Society Task Force, and the American College of Radiology. The use of computeraided detection (CADe) has been shown to improve the detection sensitivity of radiologists and to reduce their inter-observer variance. However, although standalone CADe systems can yield a high detection sensitivity, they generate a large number of false-positive (FP) detections that limits their usefulness. The FP CADe detections are largely caused by the inability of manually designed image-based features to discriminate reliably between normal variations of colon anatomy and that of true lesions. If the detection specificity of CADe systems could be improved without compromising their high detection sensitivity, this would improve their overall detection accuracy. Deep convolutional neural networks (DCNNs) have recently gained interest because they have been able to outperform conventional classifiers in terms of classification accuracy. However, reliable training of DCNNs requires the use of large annotated image databases. The problem of small number of training samples in CTC can be addressed by use of transfer learning. In this study, we compared the performance of an DCNN that is trained starting with random initial parameters (called ‘‘training from scratch’’) with that of a pretrained DCNN adjusted with transfer learning in the improvement of the performance of CADe of polyps in CTC images. Methods Materials—We sampled 154 CTC cases from a large multi-center CTC screening trial. Fecal tagging was administered in 34 % of the CTC cases, in which hydrosoluble iodine agent was administered orally without or with barium sulfate. The CTC was performed in supine and prone positions with average tube voltage of 120 kVp, tube current of 50 mAs, and slice thickness of 2.5 mm, followed by the same-day colonoscopy. Expert radiologists correlated the colonoscopy findings with those of the CTC image data to identify the location of polyps. Automated detection of polyp candidates—Polyp candidates used in this study are detected from the CTC data by use of our fully automated CADe system. The system has three main components: colon extraction, polyp detection, and FP reduction. After the colon extraction and shape-based detection of polyp candidates, the complete regions of the polyp candidates are extracted by use of a levelset method. After the calculation of several image-based shape and texture feature statistics from the extracted regions, an AdaBoost classifier is used to determine the final output of the CADe system. Deep convolutional neural networks (DCNNs)—We used the publically available Berkeley Vision and Learning Center Reference CaffeNet DCNN for the experiments. The DCNN has five convolutional and three fully connected layers. For training from scratch, the weights of the DCNN were initialized randomly. For transfer learning, we used the publically available pre-trained version of the DCNN that has been trained with millions of non-medical natural images from the Large Scale Visual Recognition Challenge 2012 (ILSVRC2012) image set. Both types of DCNNs were trained to review virtual endoluminal images of polyp candidates, which were

123

generated by placing a virtual camera on the surface of a 20-mm sphere centered at a polyp candidate. Evaluation method—The CTC cases were divided randomly into a training set and a validation set. The training set consisted of 62 cases and the validation set consisted of 92 cases with 107 biopsy-confirmed advanced neoplasms. The virtual endoluminal views of 2044 CADe polyp candidates from the training set were manually categorized into polyp and non-polyp classes, and the non-polyp class were further categorized into 8 classes: bones, folds, stool, rectal tubes, ileocecal valves, extrinsic compression, unspecific surfaces, and extra-colonic components. These categorized polyp candidates were used for training of the DCNNs. The CADe system was then used to detect polyps from the validation set, where 2005 virtual endoluminal views of the polyp candidates detected by CADe were subjected to the DCNNs to determine the final output of the CADe system. Results When the DCNN was trained from scratch, the CADe system yielded 76.6 % sensitivity at 4.9 FPs per patient. With transfer learning, CADe yielded 92.5 % sensitivity at 3.36 FPs per patient. These results suggest that transfer learning is a more effective training method of the DCNN for improving the performance of the CADe system than training from scratch. Conclusion We used the training from scratch and a transfer learning methods to adapt a deep convolutional neural network for detecting polyps from virtual endoscopic views in CT colonography and compared the performance in the improvement of the accuracy of automated polyp detection in CTC. The preliminary results suggest that transfer learning has superior performance in improving the detection performance of CADe of polyps in CTC images.

Radiomic machine learning of electronic cleansing for ultra-low-dose dual-energy CT colonography R. Tachibana1, J. J. Na¨ppi2, J. Ota3, D. Regge4, T. Hironaka2, H. Yoshida2 1 National Institute of Technology, Oshima College, Suo-Oshima, Japan 2 Massachusetts General Hospital and Harvard Medical School, 3D Imaging Research, Department of Radiology, Boston, United States 3 Osaka University, Suita, Japan 4 Institute for Cancer Research and Treatment, Turin, Italy Keywords Colon  Electronic cleansing  Dual-energy CT  Machine learning Purpose Early detection and removal of the benign precursor lesions of colorectal cancer would prevent the development of colon cancer. CT colonography (CTC) which provides a safe and accurate method for examining the complete region of the colon has been recommended by the American Cancer Society as an option for colon cancer screening. Electronic cleansing (EC) enables computer-aided detection (CAD) systems to detect polyps that are submerged in orally administered fecal tagging. However, EC methods for single-energy CTC are not able to distinguish soft tissues clearly from partial-volume tagging effects and unclearly tagged regions because of their similar CT values. Although these could be resolved using dual-energy CTC (DE-CTC), current EC methods for DE-CTC require manual ad-hoc parameter adjustments and also tend to produce subtraction artifacts [1]. In this study, we developed a radiomic machinelearning electronic cleansing (ML-EC) scheme for ultra-low-dose DE-CTC for artifact-free virtual cleansing of tagged fecal materials. Methods Figure 1 shows an overview of the proposed radiomic ML-EC scheme. First, we prepare a multi-spectral input image set that

Int J CARS consists of the acquired 80 kVp and 140 kVp CTC images, wateriodine material decomposition images, and two virtual monochromatic images at 120 keV and 160 keV (Fig. 1). Next, a machinelearning-based radiomic texture analysis is performed to generate a multi-material labeled volume with five material classes (luminal air, soft tissue, tagged fecal materials, and partial-volume boundaries between air and tagging and those between soft tissue and tagging) using either random forest (RF) [2] or deep convolutional neural network (DCNN) [3] classifiers. Finally, electronically cleansed CTC images are generated by removal of regions that were classified as non-soft-tissue materials, followed by colon surface reconstruction [1].

evaluated by two-fold cross-validation. The statistical significances of the differences of overlap ratios between different EC schemes were obtained using the paired t-test. Results The radiomic ML-EC schemes yielded higher cleansing performance than the single RF-based ML-EC scheme. For the radiomic DCNNbased ML-EC scheme, the mean and standard deviation of the overlap ratios from the 384 VOIs were 0.958 ± 0.041, which was higher than those of the single RF-based ML-EC (0.939 ± 0.060) and the radiomic RF-based ML-EC (0.949 ± 0.053). The performance differences between the single/radiomic RF-based ML-EC and the radiomic DCNN-based ML-EC schemes were statistically significant (p \ 0.001). Figure 2 shows an example of the difference between the virtual endoscopy images cleansed by the three ML-EC schemes: the RF-based ML-EC schemes (Fig. 2b, c) erroneously removed a haustral fold between air and tagging (yellow arrows), whereas the radiomic DCNN-based ML-EC preserved the fold (Fig. 2d).

Fig. 1 Overview of the radiomic ML-EC scheme To generate the multi-material labeled volume using an RF classifier, we calculate statistical features (mean, standard deviation, skewness, and kurtosis) from local volumetric 3 9 3 9 3 voxel regions of interest (VOIs) centered at each input voxel of input image volumes. The input image volumes consist of the multi-spectral image set and two gradient image volumes calculated from the water/iodine images. A radiomic feature vector (F) is generated from these statistical features at each voxel. The RF classifier is applied to F to classify each voxel into one of the five material classes. To generate the multi-material labeled volume using DCNN classifiers, region-of-interest (ROI) images centered at each input voxel are extracted from the input image volumes. After mapping the pixels of an ROI to the input layer of a DCNN, a series of convolutional and max-pooling layers are used to extract hierarchical features with increasing levels of abstraction. The top-most layer calculates the probabilities at which the input voxel belongs to each of the five material classes. We trained a separate DCNN for each multispectral input image volume. A radiomic texture analysis based on the DCNN was developed by combining the outputs of these DCNNs by a meta-classifier for classification of each voxel. For evaluation, 32 ultra-low-dose CT scans were acquired in supine and prone positions using a DE-CT scanner (SOMATOM Definition Flash, Siemens Healthcare, Erlangen, Germany). The CT slice thickness was 0.6 mm. The tube current was 25–36 mA at 140 kVp and 61–115 mA at 80 kVp. For quantitative evaluation, 384 VOIs (size: 16 9 16 9 16 voxels) representing typical sources of EC artifacts were sampled from the DE-CTC scans. The accuracies of the EC schemes were evaluated by the calculation of the overlap ratio between manually established reference standard labels and the labels obtained by the EC schemes [1]. In addition to comparing the radiomic RF-based and DCNN-based ML-EC schemes, we also evaluated the performance of a single RF-based ML-EC scheme which only used features of the single-energy 140 kVp CTC image to generate the multi-material volume. The performances were

Fig. 2 An example of the EC of a clinical ultra-low-dose DE-CTC case. (a) In the uncleansed original 2D image, the yellow arrow indicates a thin haustral fold partially covered by fecal tagging. In the virtual endoscopic images of the cleansing by (b) the single RF-based ML-EC scheme and (c) the radiomic RF-based ML-EC scheme, the fold (as indicated by the yellow arrows) is subtracted incorrectly. However, (d) the radiomic DCNN-based ML-EC scheme preserves the fold correctly Conclusion The radiomic DCNN-based ML-EC scheme that extracts optimal features automatically from input images yielded superior cleansing performance over those of the radiomic and single RF-based ML-EC schemes. The radiomic DCNN-based ML-EC scheme can be used to reveal the entire colonic surface and colonic lesions submerged in fecal materials at high accuracy without ad-hoc parameter adjustments and complicated post-processing for CAD of ultra-low-dose DE-CTC cases. References [1] Tachibana R, Na¨ppi JJ, Kim SH, Yoshida H (2015) Electronic cleansing for dual-energy CT colonography based on material decomposition and virtual monochromatic imaging. Proc SPIE Int Soc Opt Eng 9414, PMID 25844029 [2] Breiman L (2001) Random forests. Machine learning 45, 5–32

123

Int J CARS [3]

Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. The 26th Annual Conference on Neural Information Processing Systems (NIPS), 1–9

Information-preserving dual-energy image correction for improving radiomic detection of colorectal lesions in CT colonography J. Na¨ppi1, S. H. Kim2, H. Yoshida1 1 Massachusetts General Hospital, Radiology, Boston, United States 2 Seoul National University Hospital, Radiology, Seoul, South Korea Keywords Radiomics  Dual-energy CT  CT colonography  Computer-aided detection Purpose In dual-energy CT colonography (DE-CTC), the acquired dual-energy images can be degraded by pseudo-enhancement distortions that are caused by the presence of orally administered high-density fecaltagging agent in the colon [1]. Pseudo-enhancement complicates the computer-aided detection (CADe) of lesions by making it challenging to differentiate between lesions, fecal tagging, and normal anatomy based on the observed CT attenuation values [2]. It also distorts radiomic feature information that can be derived from the dual-energy images, such as dual-energy index, material density maps, effective atomic number, and spectral texture features [3]. The purpose of this study was to improve the accuracy of CADe of colorectal lesions in DE-CTC by information-preserving pseudo-enhancement correction of the dual-energy images. Methods Twenty patients were prepared for a DE-CTC examination by use of reduced one-day bowel preparation with 18 g of magnesium citrate and 50 ml of non-ionic iodine. The DE-CTC image acquisitions (SOMATOM Definition, Siemens Healthcare) were performed at 140 kVp and 80 kVp energies in supine and prone positions with 1-mm slice thickness. No intravenous contrast was used. An experienced board-certified radiologist correlated the CTC images with the findings of subsequent optical colonoscopy. The dual-energy images were corrected for pseudo-enhancement distortions by first correcting the 140 kVp image with a standard image-based correction method [2] and by next correcting the 80 kVp image proportionally to the 140 kVp correction to preserve the dual-energy information contained within the dual-energy image pair. If both dual-energy images were corrected by the standard correction method, compared with the uncorrected images the corrected water-iodine material decomposition images would show information loss (arrows in Fig. 1), whereas such image degradation is not present after the dual-energy correction (Fig. 1).

123

Fig. 1 In dual-energy CT, standard pseudo-enhancement correction of both dual-energy images distorts the dual-energy information contained within the images (see arrows in the iodine image for most obvious errors), whereas the proposed dual-energy image correction preserves the dual-energy information For evaluation, a previously developed CADe scheme that makes use of the radiomic dual-energy image information was used to detect colorectal lesions from the DE-CTC images with and without the dual-energy image correction [4]. The detection performance was assessed by use of the leave-one-patient-out evaluation. The difference in per-lesion detection performance with and without the proposed dual-energy image correction was compared by use of the recommended non-parametric method [5], where the figure of merit is based on the relative fitted area under the free-response receiver operating characteristic curve (AUFC).

Int J CARS Results There were 29 biopsy-confirmed lesions: 14 lesions measured C 10 mm and 15 lesions measured 6–9 mm in largest diameter. Approximately 55 % of the lesions were at least partially covered by fecal tagging. For all lesions (C 6 mm), the correction improved the AUFC from 0.71 [95 % confidence interval (CI): 0.52, 0.89] to 0.84 [95 % CI: 0.72, 0.97]. With the correction, the detection sensitivity was 86 % at 6 FP detections per patient on average, whereas without the correction the detection sensitivity was 72 % at 7 FP detections per patient (Fig. 2). For large lesions (C 10 mm), the correction improved the AUFC from 0.74 [0.63, 0.89] to 0.83 [0.72, 0.92]. For small lesions (6–9 mm), the correction improved the AUFC from 0.52 [0.32, 0.79] to 0.63 [0.48, 0.85]. All improvements in the detection accuracy were statistically significant (p \ 0.0001).

Fig. 2 Detection performance for all lesions with and without the proposed correction method Conclusion The proposed dual-energy image correction method can be used to correct for pseudo-enhancement distortions in all radiomic DE-CTC image data while preserving the radiomic information contained within the images. The use of the correction yields statistically significant improvement in the accuracy of CADe of colorectal lesions in DE-CTC. References [1] Na¨ppi JJ, Tachibana R, Regge D, Yoshida H (2014) Information-preserving pseudo-enhancement correction for noncathartic low-dose dual-energy CT colonography. LNCS 8676, 159–68 [2] Na¨ppi JJ, Yoshida H (2008) Adaptive correction of the pseudoenhancement of CT attenuation for fecal-tagging CT colonography. Med Image Anal 12, 413–26 [3] Heismann B, Schmidt B, Flohr T (2012) Spectral computed tomography. SPIE—The International Society for Optical Engineering [4] Na¨ppi JJ, Regge D, Yoshida H (2015) Context-specific method for detection of soft-tissue lesions in non-cathartic low-dose dual-energy CT colonography. Proc SPIE Int Soc Opt Eng 9414, PMID 25964710 [5] Chakraborty DP (2008). Validation and statistical power comparison of methods for analyzing free-response observer performance studies. Acad Radiol 15, 1554–66

Heterogeneity analysis of prostate peripheral zone lesions in 3.0 Tesla T2-weighted magnetic resonance images G. Samarasinghe1, A. Sowmya1, D. A. Moses2 1 School of Computer Science and Engineering, University of New South Wales, Sydney, Australia 2 Department of Medical Imaging, Prince of Wales Hospital, Randwick, Australia Keywords Prostate  MRI, Feature extraction  Heterogeneity Purpose Recognition of prostate cancer is important prior to treatment, especially inside the peripheral zone where the majority of tumours are present. Computer-aided heterogeneity analysis of Magnetic Resonance Images (MRI) is an effective method to discriminate abnormalities within the peripheral zone by eliminating the bias towards inter-patient and inter-scan intensity variations [1]. A classification framework for recognition of suspicious peripheral zone lesions by region-based heterogeneity analysis in prostate MRI is developed and evaluated in this study. Methods The most critical component in the proposed framework is the feature extraction methodology. Four different features were extracted for a selected lesion within the prostate peripheral zone to emphasise the lesion T2w-MRI signal intensity relative to the rest of the peripheral zone and the central gland, based on the fact that the tumours in the peripheral zone show relatively lower T2w-MRI signal intensities [1]. When deriving these features, the relative intensity distribution of the selected lesion within the peripheral zone was weighted by a function of the Euclidean distance to the particular comparing voxel or region from the geometric centroid of the lesion. The motivation behind this was that when the selected lesion is a portion of a tumour, the tumour or its lower intensity behaviour might spread to adjacent pixels. Therefore it was important to give a higher weight to the pixels that are further away but still within the peripheral zone when computing the specific intensity of the lesion. T2w-MRI datasets acquired on a General Electrical Healthcare— Discovery MR750w 3.0T MRI scanner, for 40 patients in the age range 48–73 were used in the evaluation of the features derived in the proposed model. 82 lesions were marked on 2D axial slices of the selected datasets by an expert radiologist, and based on their observations, PI-RADS [2] score (from 1 to 5) was assigned for each. In the experiment, lesions with PI-RADS score 1/2/3 were considered to be negative, and lesions with PI-RADS score 4/5 were considered as positive in determining ground-truth malignancy. There were 68 negative lesions and 14 positive lesions in total. The annotated lesions were used as training data for two different supervised classifiers: (i) a Support Vector Machine (SVM) with a linear kernel [3] and, (ii) a Random Forest Classifier (RFC) [4]. The proposed four novel features were used to train each classifier, with six different distant functions as constant, linear, logarithmic, quadratic, cubic and quartic functions of the distance to the comparing voxel or region. Leave-one-out cross validation was used to evaluate each classifier with each distance function. Results Quantitative results show that the accuracy, sensitivity and specificity depend on the distance function used in feature extraction. For both SVM and RFC and for different distance functions, the best accuracy, sensitivity and specificity were achieved when either linear or quadratic weights of distances were used to calculate the feature values. When using higher order weights for distances (cubic and quartic) sensitivity decreased significantly for both classifiers. On the other

123

Int J CARS hand, when reducing the dependency of distances by using a weakly growing weight function (logarithmic) or when eliminating the impact of the distance (constant), sensitivity seemed to be unaffected, however there were more false positives and hence specificity decreased significantly. Results of the statistical evaluation are shown in Table 1. Table 1 Statistical evaluation of the classifiers on the proposed feature model when using different distant functions Classifier Evaluation for the Accuracy Distant Function (%)

Sensitivity (%)

Specificity (%)

SVM RFC SVM RFC SVM RFC Constant

93.3

91.5 92.7

92.7

94.1

91.2

Logarithmic

96.3

91.5 92.7

92.7

97.1

91.2

Linear

98.8

98.8 92.7

92.7 100

100

Quadratic

98.8

98.8 92.7

92.7 100

100

Cubic

95.1

97.6 71.4

85.7 100

100

Quartic

95.1

97.6 71.4

85.7 100

100

Conclusion The proposed feature extraction model can be used successfully in computer-aided diagnostic assistance frameworks for recognition of prostate peripheral zone tumours. Further usage of proposed model may include combining with multi-modal MRI data, and use as a feature model for automated tumour localisation. References [1] Lematre, G., Mart, R., Freixenet, J., Vilanova, J.C., Walker, P.M. and Meriaudeau, F., 2015. Computer-Aided Detection and diagnosis for prostate cancer based on mono and multiparametric MRI: A review. Computers in biology and medicine, 60, pp.8–31. [2] Roethke, M., Blondin, D., Schlemmer, H.P. and Franiel, T., 2013. PI-RADS classification: structured reporting for MRI of the prostate. Rofo, 185(3), pp.253–261. [3] Bishop, C.M., 2006. Pattern recognition and machine learning. [4] Criminisi, A., Shotton, J. and Konukoglu, E., 2011. Decision forests for classification, regression, density estimation, manifold learning and semi-supervised learning. Microsoft Research Cambridge, Tech. Rep. MSRTR-2011-114, 5(6), p.12.

e-ASPECTS software is non-inferior to neuroradiologists in applying the ASPECTS score to CT scans of acute ischemic stroke patients S. Nagel1, D. Sinha2, D. Day3, P. Papanagiotou4, K. Fassbender5, M. Essig6, J. Heidenreich7, A. A. Konstas8, S. Gerry9, C. Roffe10, J. Hampton-Hill11, I. Q. Grundwald2,11,12 1 University Hospital Heidelberg, Neurology, Heidelberg, Germany 2 Southend University Hospital, NHS, Westcliff-on-Sea, Great Britain 3 Addenbrooke’s Hospital, NHS, Cambridge, Great Britain 4 Bremen Hospital, Department of Neuroradiology, Bremen, Germany 5 University Hospital Saarland, Neurology, Homburg, Germany 6 University of Manitoba, Radiology, Winnipeg, Canada 7 Dalhousie University, Diagnostic Radiology, Halifax, Canada 8 University of California, Radiology, Los Angeles, United States 9 University of Oxford, Centre for statistics in medicine, Oxford, Great Britain 10 North Staffordshire Combined Healthcare NHS Trust, Stroke Research, Stroke on Trent, Great Britain

123

11

Anglia Ruskin University, Clinical Trials Unit, Chelmsford, Great Britain 12 Anglia Ruskin University, Neuroscience, Chelmsford, Great Britain Keywords ASPECTS  CT  Ischemic stroke  Machine learning Purpose The Alberta Stroke Programme Early CT Score (ASPECTS) is a topographic scoring system for acute ischemic damage to brain that divides the middle cerebral artery (MCA) territory into 10 areas of interest [1]. The e-ASPECTS software is a commercially available, now CE-marked, standardized and fully automated ASPECTS scoring tool based on a machine learning algorithm [2]. In a non-inferiority trial with NCCTs of acute ischemic stroke patients from five different stroke centres we compared the scoring performance of e-ASPECTS with those of three independent neuroradiologists (NRADs), experienced in diagnosing early ischemic damage. Methods Pseudonomized baseline non contrast enhanced CT (NCCT) scans were retrospectively scored by eight different prespecified e-ASPECTS operating points (OP) as well as by three NRADs. All were blinded for any clinical information except that a unilateral ischemic stroke in the anterior circulation was suspected. The ASPECTS score of the ground truth was based on the follow up imaging and was determined by an independent core lab. e-ASPECTS is based on a machine learning algorithm. The e-ASPECTS OPs reflect settings, which essentially determine how much weight is given to misclassifications in the ‘‘true positive (damage)’’ score versus the ‘‘true negative (no damage)’’ score of the algorithm. Sensitivity and specificity as well as accuracy based on true positive (TP), true negative (TN), false positives (FP) and false negative (FN) scores were calculated individually over all ASPECTS regions and for the overall ASPECTS score for each e-ASPECTS OP and each NRAD. Since the results were expected to be correlated within patients, two methods for clustered data were used to estimate sensitivity and specificity (and 95 % confidence intervals). We then calculated the receiver-operating characteristic (ROC) curves for the region-based analysis (20*132 regions), the score-based analysis and for the dichotomized ASPECTS score of [ 5. i.e. the selection cut-off point of the ESCAPE trial. Sensitivities and specificities of the region based analysis were then compared between each e-ASPECTS OP and each NRAD. For each given couple of e-ASPECTS OP and scorers, if the lower 90 % confidence interval boundary for the difference of e-ASPECTS minus scorer did not cross -10 % for both sensitivity and specificity, then non-inferiority was concluded. Matthews correlation coefficients (MCC) which range between [-1;1] (0 = random prediction) were also calculated. Results 132 patients were included in the study and 2640 ASPECTS regions were analysed. Median age of the patients was 72 years (63–79, IQR) and 55 % were male. Mean time from onset to baseline CT was 146 ± 124 min and median NIHSS was 11 (6–17, IQR). Three patients were treated with an endovascular approach and 123 patients were treated with intravenous thrombolysis. Median ASPECTS for the ground truth on follow up imaging was 8 (6.5–9 IQR). Sensitivity, specificity and accuracy were as follows for the human scorers: NRAD1—45, 89 and 84 %, NRAD2—27, 96, 88 % and NRAD3—26, 97 and 89 %, respectively. Two e-ASPECTS OPs (#4 and#5) were statistically non-inferior to all three NRADs (all p-values \ 0.003; 44, 93, 87 and 45 %, 91 %, 85 % respectively). Importantly, e-ASPECTS OP#4 identified 97 % of patients with an ASPECTS [ 5 correctly. Finally, the Matthews correlation coefficient (MCC), used as a measure of binary classification accuracy, were best (0.36 and 0.34) for e-ASPECTS OP#4 and #5, respectively, thereby showing better prediction of the ground truth by e-ASPECTS.

Int J CARS Conclusion This is the first trial and proof-of concept study to demonstrate equivalent specificity and non-inferior or even superior sensitivity of the e-ASPECTS software compared to ASPECTS scoring by boardcertified neuroradiologists (with specific expertise in acute stroke imaging) of baseline NCCTs from acute ischemic stroke patients. A previous monocentric trial already demonstrated superiority of e-ASPECTS to stroke trainees regarding sensitivity of a region based analysis approach [3]. The strengths of this study are the multicentric design, which meant that NCCTs were acquired with various CT scanners indicating generalisation of the results. A limitation is surely that the ground truth was based on the follow up CT scan and not on expeditious MRI; however that should not hamper the overall comparison between humans and e-ASPECTS. Although e-ASPECTS is now CEmarked, careful assessment of each CT scan by the physician to rule out other pathologies and haemorrhage is still mandatory. It is not intended to replace the physician’s assessment of the scan but it can be a valuable second opinion and confirmation of the own interpretation on expert level. e-ASPECTS might be a valuable tool especially for hospitals, where expert assessment of NCCT imaging is not available 24/7. References [1] Barber PA, Demchuk AM, Zhang J, Buchan AM (2000) Validity and reliability of a quantitative computed tomography score in predicting outcome of hyperacute stroke before thrombolytic therapy. Aspects study group. Alberta stroke programme early ct score. Lancet;355:1670–1674 [2] Hampton-Till J, Harrison M, Ku¨hn AL, Anderson O, Sinha D, Tysoe S, et al. (2015) Automated quantification of stroke damage on brain computed tomography scans: E-aspects. European Medical Journal 3[1]:69–74 [3] Herweh C, Ringleb PA, Rauch G, Gerry S, Behrens L, Mo¨hlenbruch M, et al. (2015) Performance of e-ASPECTS software in comparison to that of stroke physicians on assessing CT scans of acute ischemic stroke patients. International Journal of Stroke (in press, accepted 12.Nov. 2015)

fracture usually presents a nearly rectangular shape in the sagittal plane MRI, with possible mild concavity in the superior and inferior plateaus and in the posterior and anterior walls of the vertebral body. Benign VCFs can present accentuated concavity in the vertebral plateaus and may cause posterosuperior wall fragment retropulsion with angulated or irregular contours; subchondral bone impaction can cause rough and notched contours. Malignant VCFs may also have concave deformation of the vertebral plateaus; however, their contours may be relatively rounded or smoothened due to bulging neoplastic tissue, which can also lead to convexity of the posterior vertebral body wall [1,2]. We propose image-processing methods based on measurement of the distance from a vertebral body’s contour to its centroid to recognize VCFs. We applied these methods to images in the median sagittal plane of T1-weighted MRI exams. Methods This study has been approved by the Research Committee of Ethics of the Ribeira˜o Preto Medical School. We used a set of T1-weighted MRI exams of 63 patients, including 38 women and 25 men, with the mean age of 62.25 ± 14.13 years, collected from the Radiology Information System (RIS) of our university hospital. Each patient was diagnosed with at least one lumbar VCF. Manual segmentation of the lumbar vertebral bodies was performed to define the regions of interest (ROIs) for the study. The data set includes 102 lumbar VCFs (53 benign and 49 malignant) and 89 nonfractured (normal) lumbar vertebral bodies. Euclidean distances from the centroid of a given ROI to each pixel on its boundary were computed. Figure 1 shows examples of contours of a normal vertebral body, a benign VCF, and a malignant VCF with the distances to their centroids. We computed seven features using the distances for each ROI: the mean, standard deviation, variance, coefficient of variation, skewness, kurtosis [3–5], and a combination of the fourth-order and second-order moments (mf) as defined by Shen et al. [5].

Recognition of vertebral compression fractures in magnetic resonance images using measures of distances to the centroid L. Frighetto-Pereira1, R. M. Rangayyan2, G. A. Metzner1, P. M. Azevedo-Marques1, M. H. Nogueira-Barbosa1 1 University of Sa˜o Paulo, Ribeirao Preto Medical School, Ribeira˜o Preto, Brazil 2 University of Calgary, Department of Electrical and Computer Engineering, Schulich School of Engineering, Calgary, Canada Keywords Magnetic resonance imaging  Distances to the centroid  Shape analysis  Vertebral compression fracture Purpose A common clinical problem in the lumbar spine is vertebral compression fracture (VCF), particularly in elderly patients. A benign VCF is a consequence of osteoporotic fragility and is the most common type of non-traumatic vertebral fracture in the elderly. Bone metastatic disease may cause malignant VCF that is also common in the elderly population. The recognition of VCFs and differentiation between the two types of VCFs is fundamental for appropriate treatment [1]. Currently, magnetic resonance imaging (MRI) is the most suitable and effective imaging modality for early detection and investigation for differential diagnosis of nontraumatic vertebral fractures. The shapes of vertebral bodies may show multiple types of changes after a compression fracture. A vertebral body without

Fig. 1 Examples of distances between a vertebral body’s contour and its centroid: (a) normal vertebral body, (b) benign vertebral compression fracture (VCF), and (c) malignant VCF We performed two types of classification: both types of VCFs together versus normal vertebral bodies and benign VCFs versus malignant VCFs using the k-nearest-neighbor (k-NN) method with k = 5, the naı¨ve Bayes method, and a neural network with radial basis functions (RBF network) as classifiers. Before the classification step, feature selection was performed using the wrapper method with 10-fold cross-validation. The performance of each classification procedure was analyzed using the area under the receiver operating characteristic (ROC) curve (AUROC) and accuracy (ACC) computed by using the confusion matrix. Results The AUROC values for the classification of all VCFs together versus normal vertebral bodies varied from 0.93 to 0.96. The best classifier for this type of classification was the naı¨ve Bayes, which obtained AUROC = 0.96, ACC = 89 %, sensitivity = 0.84, and specificity = 0.94. For the naı¨ve Bayes classifier, only kurtosis was not selected in the feature selection step. A high value of specificity was also obtained in three-way classification separating malignant VCFs,

123

Int J CARS benign VCFs, and normal vertebral bodies in one step: The confusion matrix for the naı¨ve Bayes classifier showed that only 3 cases of the 89 normal vertebral bodies were misclassified. The results of classification of benign VCFs versus malignant VCFs did not present good performance. The AUROC values for this type of classification varied from 0.61 to 0.72 and the ACC values varied from 59.8 % to 70.59 %. The results of the present study agree with those of our previous study using other shape measures [6]. Work is in progress on the inclusion of measures of variation of T1 signal intensity and texture, which is expected to improve the accuracy of discrimination between benign and malignant VCFs. Conclusion Shape features can provide good performance in the recognition of VCFs versus normal vertebral bodies in MRI with good sensitivity and high specificity. Further classification of VCFs as benign or malignant requires additional features representing other characteristics. Acknowledgments We thank Rafael Menezes-Reis and Faraz Oloumi for their contribution to this work. This study was funded by Sa˜o Paulo Research Foundation (FAPESP, grant numbers 2014/12135-0 and 2015/087786), National Council of Technological and Scientific Development (CNPq, grant numbers 401950/2012-3 and 306576/2014-7), Financing of Studies and Projects (FINEP, 01/2006 ref. 0184/07) of Brazil, and the Natural Sciences and Engineering Research Council of Canada (NSERC, Grant Number 43930-2012). References [1] Jung HS, Jee WH, McCauley TR, Ha KY, Choi KH (2003) Discrimination of metastatic from acute osteoporotic compression spinal fractures with MR imaging. Radiographics : a review publication of the Radiological Society of North America, Inc 23 (1):179–188. doi:10.1148/rg.231025043 [2] Cuenod CA, Laredo JD, Chevret S, Hamze B, Naouri JF, Chapaux X, Bondeville JM, Tubiana JM (1996) Acute vertebral collapse due to osteoporosis or malignancy: appearance on unenhanced and gadolinium-enhanced MR images. Radiology 199 (2):541–550. doi:10.1148/radiology.199.2.8668809 [3] Rangayyan RM (2005) Biomedical Image Analysis. CRC Press. doi:citeulike-article-id:3973195 [4] Gupta L, Srinath MD (1987) Contour sequence moments for the classification of closed planar shapes. Pattern Recognition 20 (3):267–272. doi:10.1016/0031-3203(87)90001-X [5] Shen L, Rangayyan RM, Desautels JL (1994) Application of shape analysis to mammographic calcifications. IEEE Transactions on Medical Imaging 13 (2):263–274. doi: 10.1109/42.293919

Integrating thinning and shape-based candidate extraction for automatic aneurysm detection from head MRA images S. Kudo1, Y. Nomura2, T. Nakaguchi3 1 Chiba University, Faculty of Engineering, Chiba-shi, Chiba, Japan 2 University of Tokyo Hospital, Department of Computational Diagnostic Radiology and Preventive Medicine, Tokyo, Japan 3 Chiba University, Center for Frontier Medical Engineering, Chibashi,Chiba, Japan Keywords Computer-assisted detection  Magnetic resonance angiography  Cerebral aneurysm  Thinning Purpose Unenhanced magnetic resonance arteriography (MRA) is widely accepted as a noninvasive diagnostic tool for cerebral aneurysm screening. However, detection of small cerebral aneurysms is a relatively difficult task for radiologists, and computer-assisted detection (CAD) software may support them [1]. In previous studies of

123

automated detection for cerebral aneurysms, two types of approaches have generally been utilized, which were the 3D shape-based filtering and the analysis of vessel structure based on thinning. In this study, we developed integrating thinning and shape-based candidate extraction for automatic aneurysm detection in MRA images. Methods The overall scheme is shown in Fig. 1. First, interpolation and vessel extraction are carried out as preprocessing. After that, the aneurysm candidate extraction (ACE) is carried out. The detail of the ACE is described later. Finally, a classifier based on random forest [2] is employed to determine the likelihood of an aneurysm on the basis of 27 feature values of the candidates, such as volume and statistics of voxel value, shape index [3], and dot enhancement filter based on eigenvalues of Hessian matrix (Sblob) [4].

Fig. 1 Flowchart of our detection algorithm In ACE, first, the region based on 3D shape-based filtering, named Rshape, is extracted as follows: 1) The initial region Rshape_init is extracted as a set of connected voxels with shape index 3 0.9. If shape index at the adjacent voxels of Rshape_init is higher than 0.8, the voxels is also extracted. 2) Connected component analysis is performed. If Sblob value at all voxels of each component in is zero, the component is removed. Next, the region based on thinning, named Rthinning, is extracted as follows: 1) 3D thinning is performed at the extracted vessel region in the preprocessing (Rvessel) to obtain the initial tree Tinit. 2) The short branches are removed from Tinit. 3) Dilation with a spherical kernel is carried out to obtain the region of normal vessel Rnormal_vessel. The radius of dilation is set to the estimated distance to the vessel wall. 4) Rthinning is obtained by subtracting Rnormal_vessel from Rvessel. Finally, the final candidate region extraction image is created by the following equation. RACE ¼ Rshape URthinning where U is OR operation. We used 300 cases of the 3D time-of-flight unenhanced MRA data accumulated from three 3T MR scanners (two Signa HDxt and one Discovery MR750, GE Healthcare, Waukesha, WI, USA), which was the dataset for initial training in clinical evaluation trial held in The University of Tokyo Hospital (named UTH CAD Challenge 2015) [5]. This dataset included 150 positive cases (178 aneurysms in total) and 150 normal cases. Each positive case includes at least one aneurysm of 2 mm or more in diameter, which was determined by consensual reading by two experienced radiologists, and areas of aneurysm were defined by pixel-by-pixel painting. A three-fold cross-validation was carried out to evaluate the performance of the proposed method. As comparison, we also evaluated the sensitivity for each detection method (shape index, Sblob, thinning). All evaluation were performed using a personal computer with an Intel Core i7-4790 4.0 GHz CPU, 32 GB RAM, and a Microsoft Windows 7 Professional SP1 x64 operating system. Results The sensitivity of ACE were 89.9 % (shape index only), 95.5 % (Sblob only), and 94.9 % (thinning only), and 96.0 % (proposed).

Int J CARS Figure 2 shows the free-response receiver operating characteristic (FROC) curve with error bar plot for the proposed methods using three-fold cross-validation. From Fig. 2, the average sensitivity at 5 false positives per case was 73.6 %. The times required to detection per case were 79.3 ± 12.2 s.

Fig. 2 FROC curve of the proposed method Conclusion We have developed the automated aneurysm detection in MRA images by integrating shape-based filtering methods and thinningbased methods. From the results, the detection sensitivity of the proposed method achieved higher sensitivity than that of each detection methods. References [1] Miki S, Hayashi N, Masutani Y, et al. (2016) Computer-assisted detection of cerebral aneurysms in MR angiography in a routine image-reading environment: effects on diagnosis by radiologists, AJNR, 2016 Feb 18 (Epub ahead of print) [2] Breiman L (2001) Random forests, Machine Learning, vol. 45, no. 1, pp. 5–32, 2001 [3] Dorai CC, Jain AK (1997) COSMOS: a representation scheme for 3D freeform objects, IEEE Trans Pattern Anal Mach Intell, vol.19, no. 10, pp.1115–1130, 1997 [4] Li Q, Sone S, Doi K (2003) Selective enhancement filters for nodules, vessels, and airway walls in two- and three-dimensional CT scans Medical Physics, vol.30, no.8, pp.2040–2051, 2003 [5] Multicenter study for training and evaluation of various computer-assisted detection (CAD) software (2nd period). https://upload.umin.ac.jp/cgi-open-bin/ctr/ctr.cgi?function=brows &action=brows&recptno=R000020835&type=summary&language=E. Accessed 5 March 2016

2

IIT Kharagpur, Advanced Technology Development Centre, Kharagpur, India 3 VIT University, Biomedical, Vellore, India 4 EKO CT & MRI Centre, Radiology, Kolkata, India Keywords CT  Stroke lesion  GM3RF  Intensity histogram analysis Purpose Stroke is the first cause of disability and third leading cause of death in the developed country. When there is restricted the flow of blood in the brain, it leads to stroke. On the basis of restriction of blood, flow stroke is broadly divided into two categories: ischemic and haemorrhagic. When the restricted flow of blood is due to the presence of a clot in the blood vessel it leads to ischemic stroke and when the restriction of blood flow is due to the rupture or breakage of blood vessel it leads to haemorrhagic stroke. Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) are the common imaging modality for stroke. Being economical and wide availability of CT makes it preferable over MRI. The main problem faced by radiologists in detection ischemic stroke from CT images are the overlapping intensity and poor edges of the lesion. Few researchers have tried to address this issue. In 2003 Mellunas et al. [1] has solved the problem by using local means and standard deviation. In 2009 Chawla et al. [2] have automated the classification of stroke using wavelet transform. Methods This paper introduces an automated delineation methodology for demarcating stroke lesion area from brain CT images using a hybrid model (GM3RF), which consists of Gaussian Mixture Model (GMM) and Hidden Markov Random Field (HMRF). The proposed methodology mainly consists of three steps—extraction of brain area and eliminates the skull by using intensity histogram analysis; hybridization of GMM and HMRF models for segmentation of theaffectedpixels; and finally morphological operators have been used to remove misclassified area. We have shown the comparison of automated detected lesion along with the ground truth Fig. 1.

Computer-aided diagnostic prediction of stroke lesion from Brain CT Images using GM3RF model M. Nag1, D. China2, R. Nag3, A. Sadhu4, C. Chakraborty1 1 IIT Kharagpur, School of medical science and Technology, Kharagpur, India

Fig. 1 (a1-a4): The skull stripped input image;(b1-b4): Ground Truth; (c1-c4): Detected Stroke Lesion and (d1-d4): Delineated stroke regions and their corresponding area

123

Int J CARS Results The predicted lesion is compared with the ground truth generated by the two experts with the help of Dice coefficient (DC), Jaccard score (JS) and correlation coefficient (CR). In this study, DC, JS and CR values in regard to the performance evaluation of the proposed methodology are found to be 0.79, 0.75 and 0.72 respectively based on our datasets. This model was tested on 82datasetsobtained from 22 patients. This attempt will, of course, help the clinicians for better surgical planning in case of emergency. In Table 1 we have compared our algorithms with the other existing methods proposed by different researchers.

[2]

Chawla M, Sharma S, Sivaswamy J, Kishore LT (2009) ‘‘A method for automatic detection and classification of stroke from brain CT images’’. Annual International Conference of the IEEE, pp. 3581–3584

LungCAD: an open-source software to analyze ground glass opacity tumors

Table 1 Comparison of proposed model with other existing model

B. Shah1, M. Li2, R. San Jose´ Estepar1, J. Jayender1 1 Surgical Planning Laboratory, Brigham and Women’s Hospital, Harvard Medical School, Radiology, Boston, MA, United States 2 HuaDong Hospital, Fudan University, Radiology, Shanghai, China

Authors

Keywords Lung nodules  CAD  GGO  SVM

Methods

Imaging Validation Modality

Chawla et al. (2009)

Automatic histogram and wavelet based classification

CT

Matesin et al. (2001)

Symmetry detection and seeded region growing

CT

Mellunas et al. (2003)

Local means and standard deviation intensity based segmentation

CT

Proposed

GM3RF

CT

DC:0.67; JS:0.63; CR:0.65 DC:0.54; JS:0.47; CR:0.53 DC:0.51; JS:0.43; CR:0.42 DC:0.79; JS:0.72; CR:0.75

Conclusion The proposed method proves its efficiency of segmenting the ischemic infarct lesion from the CT Image. The novelty of the work lies on the approach to prediction of the ischemic infarct. References [1] Meilu¯nas M, Usˇinskas A, Kirvaitis R, Dobrovolskis RA (2003) ‘‘Automatic contouring of segmented human brain ischemic stroke region on CT images 1.’’ Mathematical Modelling and Analysis Vol.8, no. 1, pp.43–50

123

Purpose Lung cancer is the leading cause of death worldwide, and the classification of abnormal lesions in its early stages is crucial to improve survival. Many pulmonary nodules are seen as ground-glass opacities (GGO) that are difficult to classify. Although chest computed tomography (CT) helps locate these nodules, they are typically small in size and do not have clear boundaries. Since many of them are malignant, they must be analyzed thoroughly for evaluation. To simplify the process, we set out to develop LungCAD, an open-source software that enables users to classify GGO nodules as atypical adenomatous hyperplasia (AAH), adenocarcinoma in situ (AIS), minimally invasive adenocarcinoma (MIA) or invasive adenocarcinoma (IAC). This module makes it easy for researchers to study ground-glass nodules, and so, may significantly improve the ability for physicians to diagnose them. Methods LungCAD has been developed in the image-processing and navigation software, 3D Slicer. The module has been developed in Python and uses Qt for designing the user interface. To make it practical for a variety of purposes, we have subdivided the module into three parts: lesion segmentation, lesion quantification, and GGO classification. Once the user inputs a CT image volume and a seed point, the lesion is segmented using a level set segmentation algorithm from the Chest Imaging Platform [1]. This ‘‘heterogeneity’’ of lesion mask can then be quantified in LungCAD by calling the HeterogeneityCAD module [2] on command line using 65 imaging metrics. Subsequently, the imaging metrics are passed to a Support Vector Machine (SVM) algorithm, trained with the LIBSVM package [3], to predict the lesion type (Figs. 1, 2).

Int J CARS running through the lesion. In addition, it does not seem to be affected by structures that may appear as a solid component, and so, may be effectively utilized by LungCAD. The software module has been assessed for quantifying tumor heterogeneity and computes 65 imaging characteristics of a lesion, which were then input to the SVM. The SVM used in LungCAD yielded 70.9 % accuracy for classifying the four GGO type, compared to 39.6 % accuracy by expert radiologists. Additionally, the prediction accuracy for whether a lesion is indolent (AAH and AIS) or invasive (MIA and IAC) was determined to be 88.1 %. The algorithm also differentiated between AIS and MIA lesions with an accuracy of 73.1 % compared to 35 % accuracy by expert radiologists. LungCAD successfully combines these results and displays the predicted lesion type after user has inserted a volume and seed point. Conclusion LungCAD represents one of the first open-source platforms to support the diagnosis of pulmonary GGO nodules. It integrates all of the steps needed to classify these nodules, and thus makes it easy for users to analyze the malignancy of lesions. The high accuracy achieved by the SVM could make LungCAD a valuable tool for researchers and radiologists alike. The workflow of the designed Slicer module has been described in detail, and can be considered a starting point for the development of a complete GGO identification program to be applied in a clinical setting. References [1] www.chestimagingplatform.org [2] https://www.slicer.org/slicerWiki/index.php/ Documentation/Nightly/Modules/HeterogeneityCAD [3] https://www.csie.ntu.edu.tw/*cjlin/libsvm/

Towards a lung tuberculosis CAD: selecting the CT image analysis method V. Kovalev1, A. Kalinovsky1, A. Skrahina2, A. Rosenthal3, A. Gabrielian3 1 Institute of Informatics, Biomedical Image Analysis, Minsk, Belarus 2 Scientific and Practical Center for Pulmonology and Tuberculosis, Minsk, Belarus 3 National Institute of Allergy and Infectional Diseases, NIH, Bethesda, United States Keywords Lung tuberculosis  CT  Image descriptors  CAD Fig. 1 Screenshot of LungCAD module in 3D Slicer

Fig. 2 Result of Lesion Segmentation algorithm Results The result of the segmentation algorithm was evaluated individually and found to be reliable as it prevents the segmentation of vessels

Purpose The lung tuberculosis (TB) remains a serious problem of public health in a number of regions [1]. This is because of a permanently growing incidence of drug resistant cases, the increasing rate of migration from the countries with low-level incomes to the developed ones, and due to some other factors. This work is part of a large project, which involves several countries and which supposes creating free information resources on lung TB [2] as well as development of web-based computer assisted diagnosis services (see experimental web site [3]). Thus, the purpose of this study was to compare the relevant CT image analysis methods and selecting a technique, which is suitable for describing and classification of CT images in the lung tuberculosis CAD. Methods Materials. Performance of different methods was assessed on the rather hard problem of automatic classification of five TB classes distinguished in real clinical practice. Thus, CT images of 500 TB patients representing 5 TB classes, 100 CTs for each class were sampled from CT archive containing about 9000 TB cases. The classes include: Infiltrative TB (TB-1), Focal TB (TB-2) Tuberculoma (TB-3), Miliary TB (TB-4), and Fibro-cavernous TB (TB-5).

123

Int J CARS Imaging was performed using LightSpeed Pro 16 scanner with typical slice thickness of 2.5 mm. The archive images can be freely downloaded from the TB portal [2] and viewed/explored using contentbased image retrieval engine available in [3]. Lung segmentation was performed by method reported in [4]. Methods. The three methods compared were texture analysis methods including: (a) The extended co-occurrence matrices [5] with voxel pairs considered in 3D and spacing of 1.4 mm (CO-OCC). (b) The method capitalizing on commonly known Local Binary Patterns, which was adapted for 3D (LBP-3D). (c) An original method introduced by authors, which employs binary descriptors produced by PCA-generated filters (BFPCA). The use of BFPCA method supposes performing three main steps given below. Step-1: Generation of a bank of adaptive differential filters: (a) Random sampling of cube-shaped CT image patches from a subset of input images. (b) Voxel-wise multiplication of patches by 3D Gaussian. (c) Supplying the patches to the PCA method and selecting the leading components as filters. Step-2: Creating binary filtered CT images by convolving with selected set of filters. Step-3: Generating image descriptors which are essentially the histograms of values computed by voxel-wise concatenation of binary images. Results At the experimentation stage, CT images were categorized into 5 TB classed using the above three kinds of image descriptors with the help of SVM, K-nearest neighbors (K = 5), and logistic regression classifiers. All the experiments were performed using v-fold validation with v = 10. The best results for all 3 methods were obtained with K-nearest neighbors classifier. Corresponding confusion matrices together with the values of general classification accuracy (i.e. the accuracy of simultaneous classification into 5 classes) are presented in Fig. 1.

As it can be seen from Fig. 1, the BFPCA method which employs binary descriptors produced by PCA-generated banks of filters outperforms both the 3D co-occurrence and 3D LBP methods. Example of adaptive differential filters automatically generated with the help of PCA method based on 3D patches of CT lung images of TB patients are shown in Fig. 2.

Fig. 2 Example of the first 9 leading filters automatically created by the PCA method. For each 3D filter, the 3 orthogonal sections are shown As it can be immediately inferred from the confusion matrices of Fig. 1, the five TB types tend cluster into two groups consisting of (TB-1, TB-2, TB-3) and (TB-4, TB-5) clusters respectively. This would allow to introduce a more efficient hierarchical classification procedure on the further steps of this work. Conclusion The results of present study suggest that the CT image analysis method which capitalizes on binary descriptors produced by PCAgenerated filters can potentially be employed as a basic tool for prospective CAD on lung tuberculosis. References [1] Global tuberculosis report 2014. (2014) World Health Organization, WF 300, WHO Press, ISBN 978-92-4-156480-9, 170 p. [2] http://tuberculosis.by/Last visited 12.01.2016. [3] http://imlab.grid.by/Last visited 12.01.2016. [4] Hu S, Hoffman E, and Reinhardt JM (2001). Automatic lung segmentation for accurate quantitation of volumetric X-ray CT images. IEEE Transactions on Medical Imaging, 20(6):490–498. [5] Kovalev VA, Kruggel F, Gertz H-J, and von Cramon DY. (2001) Three-dimensional Texture Analysis of MRI Brain Datasets, IEEE Transactions on Medical Imaging, 20(5):424–433.

Clinical validation of a web- and cloud-based lung computer aided detection system Fig. 1 Confusion matrices and the general accuracy scores for 3 different methods of classification of CT images of lung tuberculosis

123

A. Traverso1,2, E. Lopez Torres2,3, C. Bracco4, D. Campanella5, M. E. Fantacci6, D. Regge5, M. Saletta2, M. Stasi4, L. Vassallo5, P. Cerello2

Int J CARS 1

Polytechnic University of Turin, DISAT, Turin, Italy INFN, Turin Section, Turin, Italy 3 CEADEN, Habana, Cuba 4 Candiolo Cancer Institute-FPO, Medical Physics, Candiolo, Italy 5 Candiolo Cancer Institute-FPO, Radiology, Candiolo, Italy 6 University of Pisa, Physics, Pisa, Italy 2

Keywords CAD  Lung cancer  Cloud computing  WEB Purpose A Computer Aided Detection (CAD) system for the automated identification of pulmonary nodules in chest Computed Tomography (CT) scans is being clinically validated. Lung cancer screening using annual low-dose CT reduces lung cancer mortality by 20 % in comparison to annual screening with chest radiography [1]. Noncalcified pulmonary nodules are the early manifestation of lung cancers, the leading cause of cancer-related deaths worldwide. In CT lung cancer screening, many millions of CT scans will have to be analyzed, which is an enormous burden for radiologists. In general, the detection of pulmonary nodules is a very challenging and difficult task for radiologists due to the high number of noisy images (‘slices’) to be analyzed for each patient. Furthermore, very often, pulmonary nodules are small pathological Region of Interests (ROIs) totally embedded inside the lungs or attached to other anatomical structures, such as vessels. To support radiologists, researchers have started implementing CAD algorithms. Several studies proved the positive impact of CADs as a support for radiologists in the detection, with benefits on the overall performance [3]. Despite these very prominent results, CAD systems have not spread in clinical routine yet. In fact, the standard approach to make CAD algorithms available in the clinical routine of health facilities, that is the deployment of standalone workstations, usually equipped with a vendor-dependent Graphic User Interface (GUI), presents several drawbacks, such as the high fixed cost of the software licenses and the dedicated hardware and the rapid obsolescence of both. The diffusion of Cloud Computing solutions, accessible via secure Web protocols, solves almost all the previous two issues. In addition, the Software as A Service (SaaS) approach provides the possibility of combining several CADs, with demonstrated benefits to the overall performance [4]. Methods The group has developed a CAD for the automated identification of pulmonary nodules in chest CT scans, based on a multi-thread approach that combines the results of two independent algorithms and provides a framework for further extension to others. The CAD was recently validated on several public datasets (including the Lung Image Database Consortium [2]), with a total of 1088 CTs, providing a sensitivity of about 80 % in the 4–6 false positive findings/scan range [5]. Having demonstrated the algorithm generalization capabilities, the development team tackled the issue of making it available to the largest possible user community. Therefore, a Web/Cloud prototype was designed and implemented: CTs are uploaded through a Web front-end interface and analyzed by a dedicated cloud-backend. In the cloudbackend computation resources (i.e. Virtual Machines) are creating according to live demanded computational needs. The proposed approach implements data security by means of CT anonymisation and secure transfer protocol (https), and avoids all the issues related to the software deployment on a distributed environment. CT scans can be uploaded asynchronously by ICT staff in health facilities, while the CAD results are directly sent to a pool of radiologists, not necessary belonging to the same institution of the submitting centre, through e-mail accounts in DICOM-compatible format. In order to validate the system in clinical practice, collaboration with the radiology department of a hospital has started. Every day about 10–20 CT scans associated to oncological patients undergoing staging or re-staging is submitted to the system. A panel formed by three radiologists, with different level of expertise, independently annotates the cases through the web interface of the system in ‘first-reader’ mode. Only after having completed the

annotation, M5L CAD results are prompted to the radiologist, who has the possibility to review and assess them, with the option to include them in the annotation, reject them as false-positive findings or label as clinically non-relevant nodules. Results The preliminary results on a set of 47 scans are summarized in Table 1. In the analysis, the gold standard was defined by clinically relevant nodules (according to the malignancy score) with a diameter larger than 3 mm. The first row indicates the number of nodules annotated by the three radiologists (RAD0, RAD1 and RAD2) and the subsample of nodules annotated by all the radiologists (RAD0&RAD1&RAD2). The second row shows the average number of FP/scan of the CAD marks. The fifth row provides the number of FN of the CAD system, while row number four shows the number of findings of the CAD marked as TPs and originally not seen by the radiologists. With these data the sensitivity of the CAD with respect to the gold standard formed by the nodules found by the radiologist during ‘first-‘or ‘second-reading’ can be evaluated (row three). Sensitivity values are close to 90 %, in a range of 3–4 FP/scan range. Table 1 RAD2)

Summary table of the analysis of first 47 scans annotated by three different radiologists (RAD0, RAD1,

RAD0

RAD1

RAD2

RAD0& RAD1 &RAD2

RAD0& CAD

RAD1& CAD

RAD2&CAD

RAD0 &RAD1& RAD2&CAD

38

41

40

36

Nodules

35

38

21

17

FP/scan

3.25

4.02

3.4

3.6

Sensitivity

90 %

86 %

90 %

89 %

91 %

87 %

95 %

95 %

TP added by CAD (relative %increase)

3 (+9 %)

3 (+8 %)

19 (+90 %)

19

–

–

–

–

FN

4

6

2

2

4

6

2

2

Conclusion Preliminary results are very promising, although they must be confirmed on a significantly larger dataset, as already planned. A further preliminary analysis on 130 scans is confirming the obtained results, keeping the sensitivity values around 90 % in a range of 3–4 FP/scan. The data set under collection can be used to perform additional clinical studies, such as the validation of lung cancer risk predictor models. In addition, it would be possible to perform investigations aiming at highlighting the possible relations between morphological features of the nodules and the malignancy of the nodules themselves. The validation protocol will also be opened to other sites, after testing the system capability to handle the expected workflow. References [1] Aberle DR., Adams AM, Berg CD, Black WC, Clapp JD, Fagerstrom RM, Gareen IF, Gatsonis C, Marcus PM., Sicks JD (2011). Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med, 365:395–409. [2] Armato SG, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP et al. (2011). The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys, 38:915–931. [3] Brown M, et al. (2005). Computer-aided lung nodule detection in CT: Results of large-scale observer test1, Academic radiology 12:681–686. [4] van Ginneken B, et al. (2010). Comparing and combining algorithms for computer-aided detection of pulmonary nodules in computed tomography scans: the ANODE09 study, Medical Image Analysis 14:707–722

123

Int J CARS [5]

Lopez Torres E, et al. (2015). Large scale validation of the M5L lung CAD on heterogeneous CT datasets, Medical Physics 42:1477–1489

Table 1 continued Category

Number Meaning

Position

7

Normalized x-coordinate in the bounding box of the lung region

Evaluation of efficiency of feature values for false positive reduction of automated mediastinal lymph node detection

8

Normalized y-coordinate in the bounding box of the lung region

H. Oda1, M. Oda1, T. Kitasaka2, S. Iwano3, H. Homma4, H. Takabatake5, M. Mori4, H. Natori6, K. Mori1,7 1 Nagoya University, Graduate School of Information Science, Nagoya, Japan 2 Aichi Institute of Technology, School of Information Science, Toyota, Japan 3 Nagoya University, Graduate School of Medicine, Nagoya, Japan 4 Sapporo-Kosei General Hospital, Sapporo, Japan 5 Sapporo Minami-sanjo Hospital, Sapporo, Japan 6 Keiwakai Nishioka Hospital, Sapporo, Japan 7 Nagoya University, Information & Communications, Nagoya, Japan

9

Normalized z-coordinate in the bounding box of the lung region

Keywords Feature value selection  Lung cancer  Machine learning  Pattern recognition Purpose This paper shows the comparison result of efficiencies of feature values for false positive (FP) reduction of automated mediastinal lymph node detection. Computer-Aided Detection (CADe) system for automated mediastinal lymph node detection is strongly desired by radiologist for preventing overlooking of metastasis lymph nodes. In the previous method[1], candidate regions are obtained by intensity targeted radial structure tensor (ITRST) analysis, and FPs are reduced by SVM. For classifying each candidate region as a true positive (TP) or an FP, 30 feature values which can be categorized as geometry, position and CT value of candidate region, CT value of surrounding region, are used. However, evaluation of efficiencies of each categories has not been achieved yet. This paper compares lymph node detection performance for different sets of features utilized in FP reduction. Methods We modify our previous method presented in the Ref. [1] in the part of the FP reduction process. (1) Initial detection—Input of this method is a CT volume of the chest area. Firstly, the left and the right lung regions are segmented automatically. The processing area is identified as the region between them. We assume lymph node regions observed on CT volumes as blob-like structures. The blob-like structure enhancement filter based on Intensity targeted radial structure tensor (ITRST) analysis is utilized for obtaining candidate regions of lymph nodes. (2) FP reduction—Support vector machine (SVM) is utilized to classify each candidate region into a TP or an FP region. Feature values are calculated for each candidate region. Table 1 shows the list of feature values and their categories. Note that feature values in the category ‘‘CT value in surrounding region’’ are computed from a region made by subtracting the candidate region from dilation result (structure element is a sphere {1,2} mm) of it. Table 1 List of feature values Category Geometry

123

Number Meaning 3

1

Volume (mm )

2

Surface area (mm2)

3

Sphericity

4 5

Maximum length from contour (mm) Length of long axis (mm)

6

Length of short axis (mm)

CT value in 10 candidate region 11

CT value in surrounding region

Average Variance

12

Median

13

Maximum

14

Minimum

15

Skewness

16

Kurtosis

17, 24

Average

18, 25

Variance

19, 26

Median

20, 27

Maximum

21, 28

Minimum

22, 29 23, 30

Skewness Kurtosis

Results We evaluated the lymph node detection performance by applying the methods to the lymph nodes whose short axis is no less than 7.5 mm in 47 contrast-enhanced chest CT volumes. We utilized LIBSVM 3.17 as the SVM implementation for C-Support Vector Classification (SVC). We used the RBF kernel, and set the parameters as: the regularization parameter C = 64, the coefficient in the RBF kernel c = 0.1, the weight of TP training sample w+=1.0. Table 2 shows which categories are utilized by each feature subset, and the performance of each subset with the weight of FP training sample w- = 0.15. The symbol 4 shows that the feature category utilized in each subset. Figure 1 shows the FROC curves obtained by changing w- as {0.025, 0.05, 0.075, 0.10, 0.15, 0.20, 0.30, 0.40, 0.50, 0.60}. Table 2

Component of each subset and its performance

Subset

Geometry

1

4

2

Position

CT value in candidate region

CT value in surrounding region

4

4

4

4

4

4

4

4

3

4

4

4

4

5

4

4

6

4

4

4 4

Sensitivity(%)

FPs/case

81.7

11.9

83.3

16.0

21.7

1.3

80.0

11.5

78.3

11.5

80.0

10.6

Int J CARS

Fig. 1 FROC curve of each subset The results for the subsets 1, 4, 5 and 6 produced similar detection performance. The subset 1 utilized all categories, while the subsets 4, 5 and 6 used no feature based on CT values. In contrast, the result of the scenario 2 (no geometrical feature) and 3 (no positional feature) were deteriorated than that of the scenario 1. It can be thought that feature values of geometry and position contributed to remove FPs than those of CT values. Conclusion This paper evaluated effectiveness of types of feature values utilized in the FP reduction process. The experimental results showed that the feature values based on geometrical and positional information are useful for accurate classification and CT value-based features does not contribute so much. Our future work includes more detailed evaluation and introducing more efficient feature values to improve the accuracy of FP reduction. Reference [1] Oda H, et al. ‘‘Intensity Targeted Radial Structure Tensor analysis and its application for automated mediastinal lymph node detection from CT volumes,’’ SPIE Medical Imaging 2016, 9785–13, 2016.

Computer-aided diagnosis of breast tumors using shear wave elastography R.- F. Chang1,2,3, Y.- W. Lee1, S.- C. Chang1, C.- M. Lo4, Y.- T. Lin3, C.- S. Huang5, W. K. Moon6, M. S. Bae6, S. H. Lee6, J. M. Chang6 1 National Taiwan University, Computer Science and Information Engineering, Taipei, Taiwan, Province Of China 2 National Taiwan University, Graduate Institute of Biomedical Electronics and Bioinformatics, Taipei, Taiwan, Province Of China 3 National Taiwan University, Graduate Institute of Network and Multimedia, Taipei, Taiwan, Province Of China 4 Taipei Medical University, Graduate Institute of Biomedical Informatics, Taipei, Taiwan, Province Of China 5 National Taiwan University Hospital and National Taiwan University College of Medicine, Surgery, Taipei, Taiwan, Province Of China 6 Seoul National University Hospital and Seoul National University College of Medicine, Radiology, Seoul, South Korea Keywords Elastography  Shear wave  Breast  Computer-aided diagnosis

Purpose The stiffness of the tumor has been proved to be an important characteristic for diagnosing tumors. The malignant tumors are usually relatively stiffer than benign or normal tissues. The shear wave elastography (SWE) uses the acoustic radiation substituting the manual tissue compression to generate the stiffness of tissues which is operator independent. The purpose of this study is to develop a computer-aided diagnosis method to differentiate benign from malignant tumors using SWE images. Methods In the proposed diagnostic scheme, the tumor contour must be delineated first for the purpose of computing B-mode and elastographic features for diagnosis. The segmentation procedure follows the same steps proposed in our previous study. A series of pre-processing is conducted on the B-mode image for the best tumor segmentation. At first, to increase the grey-level difference between tumor and background regions, the sigmoid filter is utilized to enhance the contrast of the original B-mode image. Next, the gradient magnitude filter is applied to obtain the gradient image, which supplies for extracting the edge information. Furthermore, to obtain a better segmentation result, the sigmoid filter is applied again to enhance the contrast of the gradient image. The tumor contour is segmented by using the level set method from the enhanced B-mode image and mapped on the corresponding SWE image. For evaluating the possibility of a lesion to be malignant, both B-mode and SWE features are extracted for diagnosis. The B-mode features evaluate the contour, grayscale, and texture information of the tumor. The eight B-mode feature classes are shape, orientation, margin, lesion boundary, echo pattern, posterior acoustic, speckle, and gray level co-occurrence matrix (GLCM) texture feature. The first three classes depict the geometric characteristics of the segmented tumor while the other classes depict its corresponding texture information. The SWE features measure tissue strain information of the tumor and its nearby area in the SWE image. Results In this study, we used 112 biopsy-proved breast tumors composed of 58 benign and 54 malignant cases. The sensitivity, specificity and accuracy of B-mode features are 83.33, 86.21 and 84.82 %, respectively; the sensitivity, specificity, and accuracy of SWE features are 92.59, 89.66, and 91.07 %, respectively; the sensitivity, specificity, and accuracy of combine features are 96.30, 93.10, and 94.64 %, respectively. The combine feature set has the best performance among all feature sets. The Az value of B-mode feature set is 0.9661, the Az value of SWE feature set is 0.9349, and the Az value of combine feature set is 0.9151. According to the Az value, the combine features set is also higher than the other two feature sets. Conclusion Our main purpose is utilizing the B-mode, SWE and combined features to diagnose the tumor on the shear wave image. The tumor contour on the B-mode image is segmented by using the level set method and overlapped on the SWE image. The tumor contour and elasticity information are utilized to calculate the B-mode and SWE features for the diagnosis of the breast tumor. According to the experimental result, the estimated sensitivity, specificity, and accuracy based on the combined features are 96.30 % (52/54), 93.10 % (54/58) and 94.64 % (106/112). Our results suggest that combining B-mode and elastographic features has the potential to differentiate benign from malignant tumor.

123

Int J CARS

22nd Computed Maxillofacial Imaging Congress Chairman: Christos Angelopoulos, PhD (USA), Co-Chair: Yoshihiko Hayakawa, PhD (J)

123

Int J CARS

Detection of apical lytic pathoses in endodontically obturated teeth on intraoral radiographs and cone beam computed tomography studies

Keywords Computed tomography  Endodontic  Digital  Radiography Purpose To determine the diagnostic ability of Cone Beam Computed Tomography (CBCT) to not just detect but also localize apical pathoses to specific roots with pulpal necrosis, as compared to twodimensional conventional direct digital intraoral radiographs (DDR) with its known limitations [1], in order to accurately determine extent of these lesions and involvement of the cortical plates for the purpose of improving treatment outcomes. Methods Previously obtained CBCT and DDR data from 45 patients with a diagnosis of failed endodontic therapy were included, following Institutional Review Board (IRB) approval. All patients had at least two periapical radiographs (DDR) of the tooth of interest, obtained using paralleling technique and exposed with a CS 6100 sensor size #1 (Carestream Dental, Atlanta, GA, USA). All images were acquired at 12 bit depth and stored as DICOM for viewing in InVivo (Anatomage, San Jose, CA, USA). All images were histogram equalized by a trained radiologist for viewing. CBCT volumes were acquired using a CS 9000 unit (Carestream Dental, Atlanta, GA) with a limited field-of-view (FoV) and isotropic voxel size of 76 microns. All images were assigned unique, random alphanumeric identifiers. The images were analyzed by a calibrated board-certified endodontist, a senior endodontic resident in the final year of the training program, and an endodontic intern (a general dentist, in training for entry into a formal residency program). Images were analyzed by three calibrated observers for the presence of periapical radiolucencies, number and location of roots involved and any cortical erosion/perforation resulting from the lytic lesion. The readings were repeated after an interval of two weeks. Results A total of 80 roots on 45 teeth were examined using DDR and limited FoV CBCT, following a calibration session. A total of 56 roots showed apical lesions on CBCT (70 %) while only 36 apical lesions were detected using DDR (45 %) (p = 0.004) (Fig. 1). CBCT also revealed erosion/perforation of the cortices in association with16 of the roots evaluated, while only 13 of the lesions with erosion/perforation were detected via DDR. Validation was achieved at the time of surgery in all patients in order to assess the accuracy of detection.

80 Number of roots with pathoses

M. Elliott1, M.K. Nair2, U.P. Nair1 1 University of Florida, Endodontics, Gainesville, FL, USA 2 University of Florida, Oral and Maxillofacial Radiology, Gainesville, FL, USA

90

70 60 50 40 30 20 10 0 CBCT

PA

TOTAL

Fig. 1 Number of roots with detected apical osteolysis using twodimensional intraoral direct digital radiography (DDR) and Cone Beam Computed Tomography (CBCT) Conclusion CBCT images were significantly superior to DDR for detection of apical pathoses and localization of erosion/perforation of cortices with respect to roots of teeth. Results illustrate the advantages of three-dimensional imaging for localization of pathoses in diagnostically challenging cases with possible implications on treatment planning [2, 3]. Accurate localization of these lesions and a higher and accurate detection rate may result in better outcomes in retreatment cases. More studies are needed to assess the time required for full healing post endodontic treatment using the additional information gleaned from CBCT data. Future studies evaluating the diagnostic outcome on the basis of variables such as the exposure parameters, FoV, voxel size, reconstruction parameters, dose, and post-processing techniques are in order to attempt to further fine-tune the image acquisition process in CBCT for a specific diagnostic task. The effect of the training and experience of the observers on the diagnostic outcome needs to be evaluated as well. Based on the finding that 81 % of the cases had established cortical perforation as determined by CBCT and validated at the time of surgery, it is evident that histologic progression of the condition associated with failed endodontic therapy occurs in more of the cases than what was known before. In these cases CBCT would be greatly helpful in early detection where demineralization is not sufficient to show such

123

Int J CARS defects on DDR. However, it is important to remember that CBCT should be prescribed only for select, challenging cases in which information gleaned from conventional two-dimensional data precludes an accurate evaluation of the tooth of interest [4]. The ‘‘As Low As Reasonably Achievable (ALARA)’’ principle should always be considered when prescribing a CBCT or any other imaging study as dose concerns must be outweighed by the benefit of the procedure. References [1] Lofthag-Hansen S, Huumonen S, Grondahl K, Grondahl HG (2007). Limited cone-beam CT and intraoral radiography for the diagnosis of periapical pathology. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 103:114–9. [2] Nair MK, Nair UP (2007).Digital and advanced imaging in endodontics: a review. J Endod 33:1–6 [3] Rigolone M, Pasqualini D, Bianchi L, Berutti E, Bianchi S (2003). Vestibular Surgical Access to the Palatine Root of the Superior First Molar: ‘‘Low-dose Cone beam’’ CT Analysis of the Pathway and its Anatomic Variations. J Endod 29:773–775. [4] AAE and AAOMR Joint Position Statement: Use of Cone Beam Computed Tomography in Endodontics 2015 Update (2015). J Endod. Sep;41(9):1393–6.

Guided endodontics based on CBCT data T. Lambrecht1, S. Ku¨hl1, M. Zehnder1, T. Connert1 1 University of Basel, School of Dental Medicine, Basel, Switzerland Keywords Guided endodontics  CBCT data  DICOM file  Cavity preparation Purpose Obliteration of teeth may occur. Root canal treatment may be affordable due to an infection or cyst in the apical region of the tooth. However, straight access to the root canal for an effective cleaning and disinfection of the root canal is crucial for the success of the endodontic treatment. Straight access to the root canal is necessary in teeth with obliterated canals but may even end up in a via falsa. In order to avoid via falsae and in order to facilitate locating the root canals of obliterated teeth, innovative techniques such as templates for guidance may be helpful. In oral implantology, recent technologies for 3-dimensional implant planning, cavity preparation and implant insertion are based on printing a virtual template after matching a surface scan with a CBCT dataset in a software for template planification. This approach could also be transferred to the field of endodontic treatment. Methods A total of 60 extracted teeth were mounted into 6 casts (10 teeth per cast). Surface scans were performed in terms of iTero scans. Further, CBCT scans of the models were performed. The CBCT was uploaded as DICOM-file into a system for implant planification (coDiagnostiX). An individual implant mimicking a drill for endodontic treatment was designed applying the ‘‘implant designer tool’’ of the respective software. This individual device was superimposed on each tooth, in order to end up in the root canal. For template fabrication, the surface scan was uploaded as stl file and due to anatomic landmarks (e.g. the teeth) a matching of the CBCT and surface scan was performed. Based on the surface scan, a virtual template including guiding sleeves was designed for each model. The templates were printed by means of a high-precision 3D-printer (Objet Eden). The templates were mounted on the models and access cavity to the roots were performed trough

123

guidance of the sleeves. CBCT scans of all the models were performed and the deviation between the planned access cavity and the real access was determined after matching the post-treatment CBCT with the planifications. Results The mean deviations ranged between 0.16–0.21 mm (base of the cavity) and 0.17–0.47 mm at the tip. The mean angular deviation was 1.81. The methodology is based on transferring basic techniques for implant planning into the field of endodontic treatment. However, the study represents only a proof of principle and optimizations with regard to the design of the sleeves and templates are necessary for clinical application. For this, further developments and accuracy studies are necessary to confirm these first outcomes of a novel technical approach. Conclusion The present study could show that guided endodontic access cavity preparation may be performed with a high accuracy.

Clinical accuracy of pre-bending of titanium reconstructive plates on 3D printed models from CBCT for mandibular defects W. Aboelmaaty1, I. Elsharabasy2, K. Elmahdy3, N. Morgan1 1 Faculty of Dentistry, Mansoura University, Oral Medicine, Periodontology, and Oral Radiology Department, Mansoura, Egypt 2 Ministry of Health and Population, Craniomaxillofacial and Plastic Surgery Department, Mansoura, Egypt 3 Ministry of Health and Population, Oral Diagnosis and Radiology, Mansoura, Egypt Keywords Pre-bending  Titanium plates  CBCT  STL model Purpose Titanium reconstructive plates application for mandibular defects from surgical removal of various malignant tumors is considered the ideal way of treatment in such cases. The plate bending is usually done during the surgical procedure intra-operatively, which is time consuming and need a lot of effort with high incidence of error [1]. The aim of this study is to evaluate the accuracy of pre-bending of reconstructive plates on 3D models fabricated from CBCT scan in the form of full adaptation and restoration of function and esthetics. Methods Twenty cases with unilateral mandibular swelling were selected from outpatient clinic in faculty of dentistry, Mansoura University. Cone beam CT (CBCT) scan were performed with imaging protocol of 16 cm 9 8 cm FOV and 0.25 mm voxel size by iCAT next generation machine. After proper clinical investigation, radiographic assessment and biopsy taking, proper diagnosis was reached. Cases of large ameloblastoma were included in this study. Raw Dicom data form CBCT scanning were transferred to a specific image analysis software (Ondemand 3D App). After segmentation and proper positioning of facial bones, a virtual 3d model for the mandible was isolated. Then this 3d model was exported in a form of virtual STL file. The STL file was transferred to a high quality 3D printer (Stratasys object 30). A 3D model was printed (Fig. 1A, B) then titanium reconstructed plates were bent on the model (Fig. 1C, D). The plates are sterilized one day before surgery. On day of surgery, after incision and flap reflection, the prebent plate was tested in position before resection was done (Fig. 2A, B). The lesion were resected and the prebent plate was applied in place as planned (Fig. 2C, D). The accuracy of bending was evaluated by 3 oral surgeons.

Int J CARS

For radiographic assessment, interobserver and intraobserver reliability showed a high level of agreement. Conclusion We found that pre-bending of titanium plates on 3D printed models from CBCT scans were very precise and simplify the work for surgeons with time saving. References [1] Rahimov C, Farzaliyev I. Virtual Bending of Titanium Reconstructive Plates for Mandibular Defect Bridging: Review of Three Clinical Cases. Craniomaxillofacial Trauma & Reconstruction. 2011;4(4):223–234.

Direct placement of a pre-surgical designed and 3D printed implant borne dental bridge L. Verhamme1, G. Meijer1, S. Berge´1, R. Soehardi1, T. Xi1, T. Maal1 1 RadboudUMC, Oral & Maxillofacial Surgery, Nijmegen, Netherlands Fig. 1 Showing 3D printed mandibular model with posterior ameloblastoma. A, B: before titanium plate bending. C, D: Accurate titanium plate bending in position

Fig. 2 Showing surgical procedure for mandibular defect resection. A, B: Titanium plate in position after incision and flap refection. C:Autogenous bone graft in place around the plate. D: Autogenous bone and titanium plate in its planned position intraorally Postoperative CBCT scan with the same imaging protocol was done and complete clinical evaluation of the case was performed. Clinical evaluation includes: patient satisfaction, restoration of function, ethics, and proper occlusion. For postoperative CBCT radiographic assessment, interobserver and intraobserver agreements (reliability) were calculated using Kappa test. Results The prebent reconstructive plate was placed very smoothly with no error at all and shows complete adaptability. No further curvature or bending was needed at time of surgery specially for the distal part of the defect. Clinical evaluation showed complete patient satisfaction, proper restoration of function, ethics and occlusion.

Keywords Implant  Surgical template  3D printing  Dental bridge Purpose The transfer of a virtual dental implant planning to the patient using surgical templates has been extensively studied. Essential is the exact fit and stability of such a template in fully edentulous cases. Unfortunately, especially after a bone augmentation procedure, in which cases the palate is flat, rotations and translations of the surgical template are present. This hinders accurate implant placement. The purpose of this study is to present a new method that allows highly accurate implant placement followed by the placement of a pre-surgical three-dimensional (3D) printed implant borne dental bridge, per-operative, directly after implant placement. Methods Ten fully edentulous patients suffering from extreme resorption of the maxilla were treated at the Radboud University Nijmegen Medical Centre (Nijmegen, the Netherlands) for oral rehabilitation. All patients have undergone a bone augmentation procedure using iliac crest bone grafts which were fixated to the maxilla using six to eight osteosynthesis screws (2.0 mm Champy-System; KLS Martin, Tu¨ttlingen, Germany) [1]. The screws were placed perpendicular to the alveolar process in order to provide optimal support to the surgical template during implant placement. After this surgical procedure, the augmented maxillary bone was allowed to heal for a period of four to six months. Two weeks prior to implant installation, two cone beam computed tomography (CBCT) scans were obtained according to the double scan procedure. As such, the relationship between the bony structures and the patient’s relined denture was acquired. Using these scans a virtual implant planning of six implants with respect to bone volume and prosthetic demands was made. A surgical template was designed in such a way that its support is solely provided by the osteosynthesis screws, which were already inserted during the first stage surgery. Then, the surgical template was 3D (DLP) technique out of NextDentTM Surgical Guide material (NextDentTM, Vertex-Dental B.V., Zeist, the Netherlands). After 3D printing the surgical template was sterilized. Based on the implant planning, already pre-surgically, the exact implant locations are known. By combining these implant positions with the reconstructed 3D model of the patient’s denture, a temporary dental bridge was created. To meet this goal, the 3D model of the conventional denture was edited in several ways. First, the prosthetic teeth were virtually separated from the base (Fig. 1A).

123

Int J CARS

Fig. 1 Virtual adjustments to the scanned denture; A: separating dentition from the base; B: Removal of the palate and combining the planned implant positions; C: Final digital model containing base with cavities for the abutments and the dentition; D: 3D printed dental bridge Then, the palate and buccal lamella were removed to obtain a bridge-like structure (Fig. 1B). At the end, the glass spheres were removed by smoothing these out. Cavities were created at the planned implant positions to provide space for the QuickTempTM abutments (Nobel Biocare, Zu¨rich, Switzerland) with a safety zone of one millimeter (Fig. 1C). The prosthetic teeth were then 3D printed out of NextDentTM Crown & Bridge material and the base out of NextDentTM Base material. Finally, the prosthetic teeth and the base were glued together using PMMA (Fig. 1D). During surgery, all patients received general nasotracheal anesthesia. No local anesthesia was applied to avoid swelling of the palate and the alveolar process. In this way, a misfit between the palate and the surgical template is prevented. Guided by the surgical template, the positions of the osteosynthesis screws could be located easily. Hereafter, the screws were exposed and subsequently partially unscrewed to guarantee a stable position of the surgical template (Fig. 2A). According to the NobelGuideTM procedure six dental implants were installed. Hereafter, the surgical template and the osteosynthesis screws were removed. On bone level, the implants were placed in their exact position. However, with respect to the soft tissues, the implants were surrounded by alveolar mucosa. To optimize the soft tissue profile a zone of attached gingiva was transpositioned to the buccal side of the implants before installation of the 3D-printed bridge. By transecting the palatal mucoperiostal flap in an angle of 60, the palatal bone remained still covered by mucoperiosteum.

123

Fig. 2 Surgical process; A: Insertion of the screw supported surgical template; B: After implant placement and placing the Quicktemp abutments; C: Printed bridge with caps of the abutments fixed In a next step, telescopic abutments were placed (QuicktempTM, Nobel BiocareTM) onto the implants including the white plastic caps to allow immediate placement of the removable temporary bridge (Fig. 2B). To fixate the QuicktempTM abutments in the dental bridge, the pre-planned cavities were filled with a layer of SoftlinerTM (CG, Tokyo, Japan) and then placed onto the caps of the QuicktempTM abutments. After curing of the SoftlinerTM, the bridge was removed from the mouth of the patient. It was checked if the caps were properly fixed in the dental bridge (Fig. 2C). The bridge was replaced so that the patient woke up from general anesthesia with his/her dental bridge already in position. During follow-up, wound healing was monitored and all patients were interviewed about their experiences with the 3D printed dental bridge and the impact of the bridge on their esthetic appearance, tasting and speech capabilities and their quality of life. Results The 3D printed dental bridge instantly recovered the esthetic appearance of the patient. Esthetics, speech and tasting capabilities were good, restoring the self-confidence of the patient. The bridge could be easily removed by the patients themselves. Also the intraoral soft tissues appeared to heal better under the pressure of the directly placed implant borne dental bridge. Conclusion Immediate placement of a 3D printed dental bridge after implant installation is a clinically relevant method. Compared to the extra step of making a conventional temporary prosthesis, treatment time is significantly reduced and patient’s quality of life is improved in two ways. First, due to the instant recovery of the esthetic function

Int J CARS and speech capabilities, it is easier for the patients to take part in social activities. Secondly, patients tasting capabilities are not affected due to the absence of palatal coverage. Moreover, a 3D printed dental bridge immediately interconnects the installed implants; this splinting leads to favorable force transfer to the implants, thereby preventing overloading, as such ensuring the osseointegration process. References [1] Verhamme LM, Meijer GJ, Berge´ SJ, Maal TJJ (2015). The use of first stage bone augmentation screws to stabilize the surgical template in the second stage. International Journal of Oral and Maxillofacial Surgery, 44(6), 781–4. doi: 10.1016/j.ijom.2015.01.010

Application of computer-assisted navigation in mandibular reconstruction with fibula free flap J. Wu1, C. Yang1, S. Zhang1, B. Xu1, S. G. Shen1 Shanghai Ninth People’s Hospital, Shanghai Jiao Tong University School of Medicine, Department of Oral and Cranio-maxillofacial Surgery, Shanghai, China 1

Keywords Navigation  Mandibular reconstruction  Fibula free flap  Oral and maxillofacial surgery Purpose The mandible is an important component of the face, as it contributes, to motor functions and facial appearance. This article presents our experiences of computer-assisted navigation in mandibular reconstruction and evaluates its effectiveness. Methods In this study, 10 patients who underwent navigation-guided mandibular reconstruction with a fibula free flap were reviewed. According to the HCL classification [1], there were 6 cases of defect L and 4 cases of defect H in the mandibles. Computer tomography (CT) scans were performed in all patients, and the data of the skull and the fibula were transferred to the ProPlan CMF 1.4 software (Materialise, Leuven, Belgium). During preoperative planning and simulation, The ideal position for the reconstructed mandibles was determined based on mirror imaging (i.e., imaging of the unaffected side). The fibular segments were positioned to replace the resected mandible and adjusted according to the position of the mirror-image mandible and the original temporomandibular joint (TMJ). Intraoperatively, maxillo-mandibular fixation with interocclusal splints is mandatory to maintain the occlusion after segmental mandibulectomy was performed through intraoral and submandibular incisions. Then, the fibula flaps were harvested in the standard manner and shaped according to the planned measurements and 3-D printed surgical templates. When the fibula flap was transferred to the region of the mandibular defect, under intraoperative navigation, the probe was used to confirm the reconstructed angle (Fig. 1A, B) and avoid mandibular asymmetry. The position of the remnant or reconstructed condyle was also confirmed to ensure that it fit into the glenoid fossa (Fig. 1C, D). Continuous adjustments were made until the ideal positioning was achieved. The surgical results were evaluated through postoperative panoramic radiographs, coronal CT scans and image fusion. And the findings from clinical examinations, including facial symmetry, occlusion, and maximal mouth opening, were recorded at follow-up appointments.

Fig. 1 The positions of the reconstructed mandibular angle and the remnant or reconstructed condyle were confirmed by the navigation probe. (A) The probe was used to verify the position of the mandibular angle in the recipient region. (B) A screen shot illustrates that the probe’s tip, which was tracked by the navigation system, reached the target position. (C) The probe was used to verify the position of the mandibular condyle. (D) A screen shot illustrates that the navigation system was tracking the probe’s tip Results All the patients with benign mandibular neoplasms, preoperative planning, simulation and intraoperative navigation were performed successfully. The navigation system had an errors of 0.86 ± 0.21 mm after point registration. The navigation system also continuously tracked the position of the probe, which guided the positioning of the fibula flaps in relation to the preoperative planning and simulation. All of the operations were completed uneventfully. The preoperative fibula flaps were placed in their target positions using the navigation system for guidance. No serious complications occurred. Image fusion of the postoperative CT data and preoperative planning revealed that the discrepancy in the mandibular angle between the actual surgical results and the preoperative plans was 2.29 ± 1.14 mm. The panoramic radiographs and coronal CT scans illustrated that the condyles in all patients fitted into their glenoid fossae and the height of the rami are nearly the same (Fig. 2). And all of the patients were satisfied with their symmetric facial profile, normal occlusion and maximal mouth opening in the follow-up period.

Fig. 2 Surgical discrepancies were measured by fusing images of the virtual simulation and the postoperative model

123

Int J CARS Conclusion Navigation technology plays an ever-increasing role in oral and maxillofacial surgery, and has been used in procedures such as TMJ arthroplasty, tumor resection, deformity correction, craniomaxillofacial reconstruction, implantation and removal of foreign bodies [2, 3, 4, 5]. This study describes the experiences of computer-assisted navigation for accurate mandibular reconstruction. Navigation technology can provide 3-D real-time visualization of reference points and contours, and help the surgeons to position and adjust the fibular grafts intraoperatively, which may improve functional and esthetic outcomes. This technology could also be applied in mandibular reconstructions using other types of bone flap. Acknowlegments This work was supported by National Natural Science Foundation of China (81371193), Shanghai Science and Technology Committee (15441906000) and the Combined Engineering and Medicine Project of Shanghai Jiao Tong University (YG2015QN05). References [1] Jewer DD, Boyd JB, Manktelow RT, et al. Orofacial and mandibular reconstruction with the iliac crest free flap: a review of 60 cases and a new method of classification. Plast Reconstr Surg 1989;84:391–403. [2] Balasundaram I, Al-Hadad I, Parmar S. Recent advances in reconstructive oral and maxillofacial surgery. Br J Oral Maxillofac Surg 2012;50:695–705. [3] Yu H, Shen SG, Wang X, Zhang L, Zhang S. The indication and application of computer-assisted navigation in oral and maxillofacial surgery-Shanghai’s experience based on 104 cases. J Craniomaxillofac Surg 2013; 41:770–774. [4] Schramm A, Gellrich NC, Gutwald R, et al. Indications for computer-assisted treatment of cranio-maxillofacial tumors. Comput Aided Surg 2000;5:343–352. [5] Schmelzeisen R, Gellrich NC, Schramm A, et al. Navigationguided resection of temporomandibular joint ankylosis promotes safety in skull base surgery. J Oral Maxillofac Surg 2002;60:1275–1283.

developmental, or acquired, effective assessment and treatment planning and management are crucial topics between the stakeholders and need to be well addressed [2]. The specific aims of this article are threefold: (1) To conduct quantitative analysis of facial asymmetry that is in agreement with the clinical experience; (2) To justify the ‘‘perception buffer zones’’ from symmetry to serious asymmetry—an overall asymmetry index, or oAI, adapted from AI, is developed to articulate the perception [3]; (3) To construct a facial symmetry classifier [4] that would categorize the perception of facial symmetry into Perceived Normal (PN), Perceived Asymmetrically Normal (PAN), and Perceived Abnormal (PA), along with a confidence index (Ci) of the classification result. The tri-information {oAI, PN|PAN|PA, Ci} is proposed. Methods The ultimate goal of the study is to bridge the information and expectation gaps between physicians and patients. Transparent and easy-to-grasp information is essential to alleviate tensions and misunderstanding in such scenario. We perform the following schematic procedures, as shown in Fig. 1, to achieve the goal. (1) Acquire nontextual 3D CBCT/3dMDTM images. The skin texture is removed to assure that the appearance and the skin quality of the subject do not interfere with the decision of symmetry classification. (2) Locate 3D facial landmarks and compute features. Twenty 3D facial landmarks are identified, in which eight are medial and the other six pairs of landmarks are bilateral, resulting in 14 features. (3) Transform a normal non-textured face to generate 64 different degrees of asymmetrical ‘‘stimulus faces.’’ Chin and nose rotation/displacements are employed to achieve transformation. (4) Estimate rough oAI. (5) Conduct facial questionnaire surveys. All faces are categorized into either PN, PAN, or PA. (6) Build a neural network classifier that is trained to learn the classification resulted from the questionnaire analysis. Fourteen features are on the input layer and the ‘‘relative importance’’ is calculated as the weights for each feature, respectively. (7) Compute the final oAI. (8) Finally, compile the tri-information {oAI, PN|PAN|PA, Ci} and implement clinical validation.

Tri-information and classification of perceived craniofacial asymmetry S.- Y. Wan1,2, P.- Y. Tsai1, Y.- Y. Chen1, L.- J. Lo2 1 Chang Gung University, Kwei-Shan, Taoyuan, Taiwan, Province of China 2 Chang Gung Memorial Hospital, Kwei-Shan, Taoyuan, Taiwan, Province of China Keywords Craniofacial images  Facial asymmetry  Perception convergence  Facial landmarks Purpose Facial symmetry is an important clinical indicator to assess the attractiveness of appearance. Craniofacial features, shades, occlusion, skin textures, etc., can greatly affect the readings. Qualitative interpretations may easily lead to misunderstanding if without support of quantitative elaborations [1]. Further, perception and expectation gap, as well as opaque information between physicians and patients, are the major factors that lead to medical disputes. Specifically, an important Taiwan’s national health status report in 2012 showed that 26 % of domestic medical disputes were against surgical departments, which include craniofacial orthodontics and the plastic surgery, in particular. Perfect facial symmetry is only considered a theoretical existence. Most attractive people exhibit asymmetric facial nature. Even so, significant facial asymmetry can still cause aesthetic and functional problems. Whether facial asymmetry is congenital,

123

Fig. 1 Schematic Procedure Results The proposed questionnaire survey demonstrates a Cronbach’s Alpha value of 0.944, indicating its high reliability. Among all transformed faces, 14 % are categorized as PN, 23 % as PAN, 63 % as PA. The confidence index (Ci) of the classification for each stimulus face is computed in terms of the percentage of the votes, accordingly. The perceived classification results are employed to model the neural network classifier. 70 % of the results are randomly chosen to train, 15 % to validate, and 15 % to test the constructed model, achieving overall mean squared error, or MSE, of 0.999959225. The relative importance of each facial feature is calculated from the inter-layer matrix, and assigned as the parameter of its corresponding feature towards computing the oAI. The computed results show that the

Int J CARS ranking of importance of the facial features that affect facial symmetry is, in order, Al (Alar curvature), Sn (Subnasale), Ch (Cheilion), Me (Menton), Prn (Pronasale), Li (Labial, inderius), Go (Gonion), Sto (Stomion), N (Nasion), En (Endocanthion), Zy (Zygion), Ex (Exocanthion), G (Glabella), Ls (Labial, superius). Conclusion The classification results show agreement with the clinical diagnosis. The greater oAI values correspond to more obvious PA classification. The inter-category areas, PN-PAN or PAN-PA, exhibit inconsistent classifications that coincides with clinical experiences, with merely moderate confidence indices, and are identified as the ‘‘perception buffer zones.’’ The tri-information {oAI, PN|PAN|PA, Ci} can then be used to communicate between the physicians and patients to explain overall degree of a patient’s facial asymmetry, what category it would be considered by general public, and how confident the physician would render such classification. The decision whether a craniofacial surgery can then be meticulously made with sufficiently and transparently mutual understanding. Acknowledgments: This research is supported in part by grants MOST-103-2221-E-182-037 (Ministry of Science and Technology, Taiwan) and CRRPD5C0251-3, BMRP583 (Chang Gung Memorial Hospital, Taiwan). References [1] Meyer-Marcotty P et al. (2011) Three-dimensional perception of facial asymmetry. Eur J Orthod, 2011. Vol. 33, no. 6, pp. 647–53. [2] Cheong YW, Lo LJ (2011) Facial asymmetry: Etiology, evaluation, and management, Chang GungMedical Journal, vol. 34, no. 4, pp. 341–351. [3] Huang CS, Liu XQ, Chen YR (2013) Facial asymmetry index in normal young adults, Orthodontics and Cranioifacial Research, vol. 16, no. 2, pp. 97–104. [4] Chiang W-C et al. (2014) The cluster assessment of facial attractiveness using fuzzy neural network classifier based on 3D Moire´ features. Pattern Recognition, vol. 47, no. 3, pp. 1249–1260.

Statistical analysis of interactive surgical planning using shape descriptors for fibular transfer in mandibular reconstruction M. Nakao1, S. Aso1, K. Imanishi2, Y. Imai3, N. Ueda4, T. Hatanaka4, M. Shiba4, T. Kirita4, T. Matsuda1 1 Kyoto University, Graduate School of Informatics, Kyoto, Japan 2 E-growth co. ltd, Kyoto, Japan 3 Rakuwakai Otowa Hospital, Kyoto, Japan 4 Nara Medical University, Nara, Japan Keywords Fibular transfer planning  Shape analysis  Quantitative evaluation  Mandibular reconstructive surgery Purpose This study presents an improved design of preoperative planning with generalized quantitative shape indicators for fibular transfer in mandibular reconstruction. A user experiment was performed to quantitatively analyze reconstruction plans and decision-making in preoperative planning. Methods Three shape descriptors were designed to evaluate local differences between reconstructed mandibles and patients’ original mandibles. We targeted an asymmetrical, wide range of cutting areas including the mandibular sidepiece, and defined a unique three-dimensional coordinate system for each mandibular image. The generalized

algorithms for computing the shape descriptors were integrated into interactive planning software [1, 2], where the user can refine the preoperative plan using the spatial map of the local shape distance as a visual guide (see Fig. 1)

Fig. 1 Shape distance computation and mapping on the oriented fibular segments: (a) the red color shows that the fibular implants near the connection point protrudes from the patient’s native mandible, and (b) the blue color indicates depression from the patient’s native mandible Results A retrospective study was conducted with two oral surgeons and two dental technicians using the developed software. The obtained 120 reconstruction plans show that the participants preferred a moderate shape distance rather than optimization to the smallest. We observed that a visually plausible shape could be obtained when considering specific anatomical features (e.g., mental foramen, mandibular midline). Conclusion This work introduces the shape distance for quantification of local differences between planned mandibular reconstruction and the patient’s native mandible. The preoperative plan can be refined using the three-dimensional map of the shape distance. Use of the map for visual guidance will assist fine adjustment of the fibular segments. The proposed descriptors can also be used to multilaterally evaluate reconstruction plans and systematically learn surgical procedures. References [1] Nakao M, Hosokawa M, Imai Y, Ueda N, Hatanaka T, Kirita T, Matsuda T (2015) Volumetric fibular transfer planning with shape-based indicators in mandibular reconstruction. IEEE Journal of Biomed Health Informatics, 19 (2): 581–589. [2] Aso S, Nakao M, Imanishi K, Imai Y, Ueda N, Hatanaka T, Kirita T and Matsuda T, (2015) A study on semi-automatic fibular transfer planning in mandibular reconstruction, Proc. Medical and Biological Imaging: 6.

Quantitative assessment of craniofacial surgery in children with craniosynostosis via 3D scanning and statistical shape analysis N. Rodriguez-Florez1,2, M. Tenhagen1,2, O. Goktekin1,2, J. L. Bruse3, A. Borghi1,2, F. Angullia1,2, J. L. O’Hara2, G. James1,2, M. J. Koudstaal4, D. Dunaway1,2, N. U. O. Jeelani1,2, S. Schievano1,3,2

123

Int J CARS

1

UCL Institute of Child Health, London, Great Britain Great Ormond Street Hospital, Craniofacial Unit, London, Great Britain 3 UCL Institute of Cardiovascular Science & Great Ormond Street Hospital, Centre for Cardiovascular Imaging, London, Great Britain 4 Erasmus University Medical Center, Maxillofacial Surgery, Rotterdam, Netherlands 2

Keywords 3D imaging  3D shape analysis  Craniofacial surgery  Craniosynostosis Purpose In children with craniosynostosis, one or more cranial sutures fuse prematurely resulting in abnormal skull growth. This can lead to functional and/or aesthetic problems requiring surgical correction. The objective assessment of head shapes is a long standing challenge in the management of craniosynostosis. Traditional methods include pre- and post-operative computed tomography (CT) scans of the head followed by cranial index measurements. However, CT delivers ionizing radiation and requires general anaesthesia in young children; therefore, its use is confined to complex cases of craniosynostosis. Since diagnosis of single suture synostosis can be done based on 2D X-rays, these infants do not usually undergo CT scanning. In these cases, 3D optical surface imaging could provide a noninvasive radiation-free and anaesthetic-free tool to improve surgical planning and evaluation of surgical outcome. The aim of this study is to i) establish a protocol to capture 3D head shapes preand post-operatively using a 3D handheld scanner, to then ii) quantitatively asses mean head shape changes induced by craniofacial surgery using a non-parametric Statistical Shape Modelling technique. As a case study, this method was applied to patients with single suture synostosis, metopic and sagittal synostosis, who did not benefit from CT imaging. Metopic synostosis leads to trigonocephaly and can be corrected by fronto-orbital remodelling (FOR) [1]. Sagittal synostosis results in long and narrow heads and can be treated by spring-assisted cranioplasty (SAS) [2]. We hypothesise that the handheld scanner allows capturing and quantifying 3D head-shapes of infants undergoing FOR and SAS and that this method will be generalizable to other craniofacial interventions. Methods This is a prospective study, including five metopic patients (male, mean age 18 ± 2 months) who underwent fronto-orbital remodelling and nine sagittal patients who underwent spring-assisted cranioplasty (male, mean age 5 ± 1 months) at Great Ormond Street Hospital for Children (London, UK). Informed consent was acquired at the preoperative assessment. 3D scans were taken pre- and post-operatively in theatre using a structured light handheld scanner, M4D Scanner (Rodin4D, Pessac, France) in conjunction with VXelements software (Creaform, Levis, Canada). The scans were exported as STL files for post-processing. Artefacts and objects out of the region of interest were cleaned up using MeshMixer (Autodesk Inc.,www.meshmixer.com). Images were cut by a plane defined by facial anatomical landmarks using 3D voxel imaging software (Robin 3D 2006) in order to capture the surgically remodelled area (Fig. 1). Rigid registration was used to register pre- to post-operative scans as well as all pre- and all postoperative scans for each intervention. The correct registration of the scans was of paramount importance for the subsequent 3D shape analysis.

123

Fig. 1 Regions that are remodelled in surgery are chosen by cutting the 3D scans with planes defined by facial anatomical landmarks for FOR and SAS patients. The defects around the ears are filled for subsequent shape analysis A non-parametric Statistical Shape Modelling approach that does not require manual landmarking was used to calculate the 3D mean head shape before and after surgery for each patient group (Deformetrica, www.deformetrica.org) [3]. Surface distances between mean pre- and post-operative scans were calculated using VMTK ( www.vmtk.org) and visualised in ParaView (www.paraview.org). Results 3D scans of infants undergoing FOR and SAS were acquired successfully with the handheld scanner. Best images of the calvarial shape were obtained by fitting a white nylon stocking (Beagle Orthopaedic, UK) on the head of the patient. Mean pre- and post-operative 3D head shapes captured the global and regional shape changes achieved in theatre in FOR and SAS surgery (Fig. 2). In patients undergoing FOR, the metopic ridge decreased by about 3 mm and the fronto-parietal area widened by 5 mm. In patients undergoing SAS, the apex of the head widened by about 5 mm and shortened by 2 mm reducing frontal and occipital bossing.

Fig. 2 Mean pre-and post-operative 3D head shapes for metopic patients undergoing fronto-orbital remodelling (FOR) and sagittal patients undergoing spring-assisted cranioplasty (SAS). Colour maps quantify surface distances between post and pre-operative scans, positive numbers representing an augmentation from pre- to post-op

Int J CARS Conclusion In this pilot study, single suture craniosynostosis patients were scanned with a 3D handheld scanner pre- and post-operatively. The adopted statistical shape analysis method was able to calculate the mean pre- and post-operative head shapes for each intervention, thus allowing a population based analysis of FOR and SAS. 3D surface distance maps of the average scans suggest that FOR successfully improves the trigonocephaly of metopic patients, while SAS widens and shortens the skull of sagittal patients. Results proved that 3D optical surface imaging is a safe method to evaluate surgical outcomes of front-orbital remodelling and spring assisted cranioplasty. 3D photography is especially helpful for patients that have no indications for CT and it could also be implemented for other craniofacial and reconstructive interventions with the advantages of not using ionizing radiation or additional general anaesthetics. Combining 3D handheld scanning with statistical shape modelling allows calculation of the 3D mean head shape of a patient population before and after surgery, and thus allows evaluating the ‘‘average’’ outcome of a specific surgical technique. In addition, the mean model of shape change induced by surgery could help parents better understand the procedure. We conclude that non-invasive and radiation-free 3D photography followed by 3D shape analysis provides an attractive and powerful diagnostic tool for quantitative assessment of global and regional shape changes achieved in craniofacial surgery. References [1] James G., et al., ‘‘A bandeau abandoned’’, an alternative frontoorbital remodelling technique: report of 328 cases. International Society of Craniofacial Surgery 2015. Tokyo, Japan. [2] van Veelen M.L. and Mathijssen I.M., Spring-assisted correction of sagittal suture synostosis. Childs Nerv Syst, 2012. 28(9): p. 1347–51. [3] Durrleman S., et al., Morphometry of anatomical shape complexes with dense deformations and sparse parameters. Neuroimage, 2014. 101: p. 35–49.

fibrous tissue (callus) fills in the gap, which will later turn into bone. This method of bone lengthening is called distraction osteogenesis. The purpose of this project was to investigate the mechanics of mandibular distraction by means of numerical modelling. Data in the literature show how mechanical cues drive the process of mandibular remodeling and different distraction protocols show different performance due to force distribution exerted on the mandible [2]. Other groups have in the past produced mathematical models for mandibular distraction osteogenesis (MDO). Results and methodologies are available in the literature [3, 4]. In this study a retrospective population study on patients having undergone mandibular distraction was performed, in order to provide important insight into the mechanobiology of MDO. Methods For the current project, the patient database of the section of Oral and Plastic Surgery of Boston Children’s Hospital (Boston, MA) was analyzed to find suitable cases for the study: out of over 262 included in the CMF database, 28 patients had undergone mandibular lengthening by means of distraction osteogenesis between 1998 and 2015. Out of these patients, 5 patients matching study criteria (availability of historical CT performed within 12 months of the procedure, unilinear distraction, sufficient axial resolution, availability of information regarding exact position of the distractor) were analyzed. CT images were retrieved from BCH’s computer system and anonymized. They were processed in ScanIP (SIMPLEWARE, Exeter, UK) to retrieve 3D information of the patient anatomy. Information from the orthopantomogram (OPG), lateral X-ray or surgical planning was used to retrieve the location of the cut from the surgery (Fig. 1). The distraction was simulated by means of numerical modelling: the callus CAD model was imported into a commercial finite element modeler (ANSYS Mechanical, Ansys, Pennsylvania) and the distraction was simulated using realistic boundary conditions replicating unilinear distraction (0.5 mm distraction twice a day). Ossification of the callus in response to mechanical stimuli was simulated using a bone remodeling theory, theorized by Claes and Heigele and implemented by Claes Lauzen [5]. In-house code was produced to implement this model in an iterative fashion. Distraction process was simulated over the first 15 days.

Finite element modeling of mandibular distraction osteogenesis: a population study A. Borghi1, S. Schievano1, M. Koudstaal2, D. Dunaway1,3, B. Padwa4 1 University College London, Institute of Child Health, London, Great Britain 2 Erasmus MC, Department of Oral and Maxillofacial Surgery, Rotterdam, Netherlands 3 Great Ormond Street Hospital, Craniofacial Unit, London, Great Britain 4 Boston Children’s Hospital, Oral and Plastic Surgery, Boston, United States Keywords Biomechanics  Mandibular distraction  Finite element  Remodeling Purpose Craniofacial macrosomia (CMF) is a congenital condition affecting 1 in 5,000 live births that presents asymmetry of the craniofacial skeleton—particularly affecting the mandible—caused by differential growth [1]. Current surgical strategies for the management of the deformity include internal distraction, which is used to correct mandibular deformity. In this procedure, an incision is made through the skin under the jaw line, bony cuts (osteotomies) are performed to separate the mandible into two and an internal distraction device (distractor) is attached to either side of this separation. The distractor is then activated regularly to increase the bone separation between the two parts of the mandible. During the resting time between each activation, the patient’s bone grows to fill the space: at the beginning

Fig. 1 Segmentation of medical images (left) and processed 3D model with location of the cut and position of the callus, in green (right) Results The results were processed in terms of bone concentration over time. Figure 2 shows a comparison of the ossification patterns of the patients 1 and 5 within the first 15 distractions. The results show how the bone geometry has a major effect on the pattern of ossification during the early time. It was found that the percentage of ossification of the callus increases exponentially over the first few distractions and then increases linearly. There is a large inter-patient variability of bone formation over the first 4 days (distractions 1–8) which decreases substantially afterwards, with ossification patterns becoming very similar throughout the population afterwards.

123

Int J CARS

Mandibular finite element analysis after partial alveolar ridge resection due to progressive osteoradionecrosis

Fig. 2 Simulation of callus lengthening and bone formation along the distraction process Comparison with the literature show good qualitative comparison with histological data from other publications and the average values of ossification found in this study at day 15 (24.36 % ± 0.81 %) is close to the values found in porcine distraction after 12 days (23.80 % % ± 3.00 %). Conclusion In the past, works in the literature have used mechano-regulation models to assess bone regeneration within the callus of osteotomized mandibles in isolated cases [3, 4]: although important insight into tissue differentiation and callus ossification was provided, the possibility of inter-individual variability was neglected and geometrical effect wasn’t analysed. In the current study, the initial stage of distraction in 5 different patients was simulated and the effect of osteotomy location and mandibular geometry was investigated. The results show how the callus ossifies in a patient-specific way during the early distraction while the bone behavior becomes much more homogeneous over the population at a later stage. Percentage of ossified bone compared well with literature data relative to animal tests. Further model refinements will allow simulation of the whole distraction procedure; extension to a larger population derived from the BCH database will provide further insight and will allow comparison between different cases. References [1] Pluijmers BI, Caron CJJM, Dunaway DJ, Wolvius EB, Koudstaal MJ (2014) Mandibular reconstruction in the growing patient with unilateral craniofacial microsomia: a systematic review. Int. J. Oral Maxillofac. Surg. 43(3): 286–95. [2] Peacock ZS, Tricomi BJ, Lawler ME, Faquin WC, Magill JC, Murphy BA, Kaban LB, Troulis MJ (2014) Skeletal and soft tissue response to automated, continuous, curvilinear distraction osteogenesis. J. Oral Maxillofac. Surg. 72(9): 1773–87 [3] Boccaccio A, Prendergast PJ, Pappalettere C, Kelly DJ (2008) Tissue differentiation and bone regeneration in an osteotomized mandible: a computational analysis of the latency period. Med. Biol. Eng. Comput. 46(3): 283–98 [4] Reina-Romo E, Go´mez-Benito MJ, Sampietro-Fuentes A, Domı´nguez J, Garcı´a-Aznar JM (2011). Three-Dimensional Simulation of Mandibular Distraction Osteogenesis: Mechanobiological Analysis. Ann. Biomed. Eng. 39(1): 35–43 [5] Simon U, Augat P, Utz M, Claes L (2011) A numerical model of the fracture healing process that describes tissue development and revascularisation. Comput. Methods Biomech. Biomed. Engin. 14(1): 79–93

123

C. Kober1, B.- I. Berg2, C. Hellmich 3, R. Sader4, G. Kjeller5 1 HAW Hamburg, Hamburg, Germany 2 HFZ Basel and University Hospital Basel, Mund-, Kiefer- und Gesichtschirurgie, Basel, Switzerland 3 TU Vienna, Institut fu¨r Mechanik der Werkstoffe und Strukturen, Vienna, Austria 4 University Frankfurt am Main, Mund-, Kiefer- und Plastische Gesichtschirurgie, Frankfurt am Main, Germany 5 The Sahlgrenska Academy, Department of Oral & Maxillofacial Surgery, Gothenburg, Sweden Keywords Mandible  Osteoradionecrosis  Finite element analysis  Alveolar ridge resection Purpose Progressive osteoradionecrosis (ORN) of mandibular bone is a serious side effect of craniofacial oncologic treatment. Often, surgical resection seems as the only means to stop this pathology. However, due to the anatomical changes of the organ caused by the resection, its physiological load carrying behavior is severely changed. With regard to the fundamental adaptivity of skeletal tissue to mechanical loads, it is questionable how this will influence the further condition of the skeletal structure. Therefore, this abstract is dedicated to finite element analysis (FEA) of a human mandible after partial resection of the alveolar ridge due to progressive ORN. Methods FEA is performed for a case of progressive ORN over 4 years, namely from 2009-10 to 2013-10 (6 follow ups, Dentascan, GE Medical Systems). The patient (male, 47 Y) was subjected to high dose RT in 2009-01/02 due to a left-sided T4 tonsil carcinoma. In 2011-08, partial resection of the left alveolar ridge was performed due to severe ORN there. The mandibular anatomy after this resection was the starting point for the simulation. From 2011-11, also affection of the mandibular ramus was observed with, at the same time, progression towards the chin. Final resection of the entire alveolar ridge and ramus of the affected side had to be performed in 2014-11. Tetrahedral FE model was built from CT data based on the situation after partial resection in 2011-08. Inhomogeneity of skeletal tissue characterized by reduced bone quality due to ORN was respected according to [1]. At this stage of research, we refrained from respecting tissue anisotropy as the course of anisotropic trajectories (possibly changed due to ORN and resection) has not yet been resolved for pathological cases of this kind. For the underlying modular simulation concept, we refer to [2]. The temporomandibular joints (TMJ) were embedded in simplified TMJ capsules where the condyles are somehow freely mobile. Individual traction of masseter, temporal and medial pterygoideus muscles was included by force vectors according to information from the CT data. Muscle forces were reduced ipsilaterally at the affected side. The same was applied for biting forces. Finite element simulation was performed using the FEM code Kaskade, developed at ZIB Berlin. With regard to its significance for bone adaptivity, we analyzed volumetric strain as well as, for demonstration purposes, mandibular deformation exaggerated by a factor of 200 (Figs. 1a, 2a).

Int J CARS

Fig. 1 Correspondences of simulated strain profile and condition of buccal cortical bone, (a) FEA of standard biting situation, based on the situation in 2011-08, arrows indicate elevated tensile strain, (b–e) visualization of cortical bone, arrows indicate progressive cortical destruction

progression of destructive processes (Figs. 1b–e, 2b–e). Notably, during the year after resection, also the mandibular shape and the course of the alveolar ridge changed, namely a medial rotation of the ipsilateral mandibular ramus occurred which can be observed to some extent in Fig. 2e. In 2014-11, entire resection of alveolar arch and ramus were necessary. Conclusion The observed correspondences of the condition of the mandibular bone and the simulated strain profiles lead to the suggestion that the alterations of physiological mandibular biomechanics due to resection again accelerate ORN related destructive processes of the mandibular bone, possibly in form of a self-energizing process. Notably, the FEA reports only a snapshot of the biomechanical situation at the stage directly after the resection. This motivates further research about the relief of these effects e.g. by tailor-made implants for mechanical mandibular support. There, mandibular FEA has again the potential to provide valuable help. References [1] Hellmich C, Kober C, Erdmann B. Micromechanics-based conversion of CT data into anisotropic elasticity tensors, applied to FE simulations of a mandible. Ann Biomed Eng. 2008;36(1):108–22. [2] Kober C, Hellmich C, Stu¨binger S, Zeilhofer HF, Sader R. ‘‘Anatomical simulation’’ of the biomechanical behavior of the human mandible. Int J Comput Dent. 2015;18(4):333–42.

Midline-guided occlusion optimization for three-piece digital dental articulation J. Li1, C.- M. Chang1, F. Ferraz1, S. Shen1, Y.- F. Lo1, P. Yuan1, J. Zhang2, J. Gateno1,3, X. Zhou2, J. J. Xia1,3 1 Houston Methodist Research Institute, Oral and Maxillofacial Surgery, Houston, United States 2 Wake Forest University, Radiology, Winston-Salem, United States 3 Weill Cornell College of Medicine, New York, United States Keywords Craniomaxillofacial surgery  Digital dental articulation  Three-piece  Midline-guided occlusion

Fig. 2 Lingual correspondences of simulated strain profile and condition of cortical bone, (a) FEA of a standard biting situation, arrows indicate elevated compression, (b–e) lingual visualization of cortical bone, arrows indicate progressive cortical resorption For the sake of a comparison, the condition of the mandibular cortical bone was visualized by direct volume rendering based on the CT data using a specially designed transfer function with a physical color scale (Figs. 1b–e, 2b–e). Results FEA reveals severe changes compared to physiological load carrying behavior. Stress/strain profiles are highly asymmetric with especially high ipsilateral TMJ load. Buccally, high tensile strain (red color in Fig. 1a) is observed which is in correspondence with severe cortical destruction there (Fig. 1b–e). Lingually, elevated unphysiological compressive strain (blue color in Fig. 2a) is stated corresponding to continuing bone resorption at that location (Fig. 2b–e). Starting lingual bone resorption has already been observed before resection whereas buccal cortical destruction probably was severely impaired after resection. Visualization based on the follow up CT data showed further

Purpose Craniomaxillofacial (CMF) surgeries require extensive presurgical planning. It is well-known that the traditional surgical planning methods are problematic [1, 2]. The development of computer-aided surgical simulation (CASS) is aimed to solve these problems. One of the critical steps in CMF surgical planning is to reestablish a correct dental occlusion (a unique relationship between the upper and lower teeth). The physical action of articulating upper and lower casts into maximum intercuspation (MI) is relatively easy and accurate. However, it becomes complex in the virtual world, where the digital dental models are two three-dimensional (3D) images that lack collision constraints [3]. It becomes even more difficult when an upper dental arch is segmentalized into 3 individual segments—one anterior and two posteriors. Therefore, the purpose of this study is to significantly develop and improve midline-guided occlusion optimization (MGO) approach for automated 3-piece dental articulation. Methods MGO approach consists of three major steps. The first step is to extract feature points for occlusal and cutting surfaces [4]. The 3D models of the 3-piece upper dental model and the intact lower model are represented by closed mesh surfaces. The feature points represent the curves for anterior, right and left posterior segments, and lower dental arch. They are the effective areas for dental articulation. The second step is to align the anterior segment using ergodic midline match algorithm. We use clinical rules to guide the anterior alignment, i.e., upper and lower midline alignment; appropriate overjet, overbite and inclination between the upper and lower

123

Int J CARS incisors; and the curve alignment of the upper and lower arches. We first approximate the anterior upper segment and the lower arch alignment by fitting the anterior occlusal feature points to the lower ones in least square (Fig. 1(a)). The anterior segment alignment procedure is a sequential process. The upper and lower incisal midpoints (the midpoints of the right and left central incisors) are registered together. The curve for upper segment is then rotationally best fit to the lower one around the registered midpoints.

Fig. 1 Schema of ergodic midline-match transform; (a) initial alignment of the anterior upper segment to the lower arch with the upper and lower incisal registered in a given transform; (b) midline match in a given transform; (c) ergodic transforms An iterative computation is performed to generate a series of anterior segment alignment which all within the clinical normal range. For each combination (iteration) of a given overjet and overbite, the anterior segment is only translated to a new position along the sagittal plane (Fig. 1(b)). Then, a penetration adjustment is applied to detect and eliminate any possible penetrations between the upper anterior segment and lower teeth. After the adjustment, the inclination of the upper and lower incisors should still be within the normal range. If not, the resulted alignment of this combination will no longer be a candidate. The iterative computation results in a series of candidates for anterior segment alignment (Fig. 1(c)). Sums of distances between the posterior segments and the lower dental arch are calculated for each candidate. They are ranked in order and the top 5 optimal anterior segment alignment are automatically selected based on smallest sum of the distances. The last step is to align the right and left posterior segments to the lower teeth in sequence using an improved Iterative Surface-based Minimum Distance Mapping (ISMDM) optimization algorithm [4]. It combines with occlusal plane transformation, constraints for deviations among the 3 pieces, and adaptive constraints parameter adjustment. Results A total of 15 sets of patient digital dental models were randomly selected from our digital archives to validate our approach [IRB(2)#1011-0187x]. Both qualitative and quantitative results showed that all the 3-piece alignment achieved with our approach automatically were at least as good as surgeon’s hand-articulated alignment (ground truth). Figure 2 shows an example of two articulated models.

Fig. 2 Examples of algorithm-generated 3-piece articulation. Yellow: reference model; Blue: experimental model. From left to right: right oblique, left oblique and posterior views. (a) Example #1; (b) Example #2

123

Conclusion The results showed our automated MGO approach is effectively, accurate and liable. The results also demonstrated our approach’s significant clinical impact and technical contributions. The clinical contribution of this project is to allow clinicians to digitally articulate 3-piece dental models for CASS planning. The computerized plan can then be transferred to the patient in the surgery using CAD/CAM surgical splints [2]. The technical contribution is that our robust ergodic midline-match algorithm can ensure the upper and lower dental midlines are automatically aligned to each other following the clinical standard. Additionally, our MGO approach further applies an improved ISMDM aligning the two posterior segments of the upper model to the lower model to ensure they are in MI. References [1] Bell WH (editor): Modern practice in orthognathic and reconstructive surgery: Volume 1. W B Saunders Co, 1992. [2] Xia JJ, Gateno J, Teichgraeber JF New clinical protocol to evaluate craniomaxillofacial deformity and plan surgical correction. J Oral Maxillofacial Surg, 67(10):2093–106, 2009. [3] Chang YB, Xia JJ, Gateno J, Xiong ZX, Zhou XB, Wong STC: An automatic and robust algorithm of reestablishment of digital dental occlusion. IEEE Transactions on Medical Imaging, 29(9):1652–1663, 2010. [4] Li JF, Ferraz F, Shen SY, Lo YF, Zhang XY, Yuan P, Tang Z, Chen KC, Gateno J, Zhou XB, Xia JJ Automated Three-Piece Digital Dental Articulation. MICCAI, 2015.

Improvement of facial soft tissue simulation accuracy following orthognathic surgery for patient with Class III dentofacial deformity D. Kim1, C.- M. Chang1, M. A. K. Liebschner2, X. Zhang1, S. Shen1, P. Yuan1, G. Zhang3, X. Zhou3, J. Gateno1,4, J. J. Xia1,4 1 Houston Methodist Research Institute, Department of Oral and Maxillofacial Surgery, Houston, United States 2 Baylor College of Medicine, Houston, United States 3 Wake Forest University School of Medicine, Winston-Salem, United States 4 Weill Medical College of Cornell University, Department of Surgery (Oral and Maxillofacial Surgery), New York, United States Keywords Orthognathic surgery  Soft tissue simulation  FEM  Tissue forces Purpose Simulating facial soft tissue changes following osteotomy is one of the most important steps in orthognathic surgical planning. Some progresses have already been made using three-dimensional imaging techniques. However, it is still problematic in the regions of lips and chin [1], where are critical to the orthognathic surgery. Therefore, it is necessary to improve the simulation accuracy so the simulation results can be used clinically. We hypothesized that tissue forces may play an important role in the simulation of the lips and chin areas. In order to test our hypothesis, we developed a new two-step finite element method (FEM) soft tissue simulation approach to improve the simulation accuracy in the lip and chin regions. Methods Preoperative (Preop) and postoperative (postop) computed tomography (CT) scans of 14 patients with Class III dentofacial deformity were used. In the first step of our method, preop patient-specific FEM model was generated using our previously developed FEM template method [2]. The face was then divided into 10 sub-regions anatomically using cephalometric landmarks (Fig. 1). The facial soft tissue changes following the bony segment movements were simulated using traditional FEM. During the FEM simulation, Young’s modulus

Int J CARS was set to 3000, and Poisson’s ratio was set to 0.47. In the second step, sliding effect of cheek mucosa around the dental alveolar regions was implemented by considering only the parallel nodal force on the corresponding areas. Lower lip and chin forces were also applied to further improve the simulation accuracy. The lower lip force is a group of normal forces applied to the lower lip and the labiomental fold, while the chin force is a group of tangent forces applied to the chin button. In addition, a group of 2-mmthick virtual surgical plates were also installed because these plates were remained in postop CT scans and would also affect the facial soft tissue geometry. During the simulation, each force parameter was adjusted within a specified range to simulate lower lip and chin changes iteratively. The lip force parameter varied from 1000 to 7000 with interval of 1000 and the chin force parameter varied from 0 to 2000 with interval of 500. Every combination of the force parameters was tested. In order to evaluate the simulation accuracy, the simulated mesh resulted from both the traditional FEM and our two-stage approach were compared to the actual postop facial soft tissue mesh. Errors between the two groups were calculated. For our two-stage approach, the smallest error was used and its corresponding combination of the force parameters was recorded for further parameter optimization. Finally the simulation errors resulted from the traditional FEM and our two-stage approach were compared using Repeated Measures Analysis of Variance.

Table 1

Error of the first and the minimum error of the second FEM (mm)

Method

Region

Traditional FEM

Mean

1.59

1.25

1.24

1.34

1.51

1.58

1.69

2.23

1.91

2.35

2.04

SD

0.25

0.27

0.23

0.31

0.38

0.44

0.52

0.62

0.69

0.93

0.89

Mean

1.48*

1.26

1.19

1.31

1.53

1.51

1.7

2.08

1.54*

1.95*

1.53*

SD

0.28

0.3

0.23

0.35

0.5

0.48

0.61

0.85

0.63

0.83

0.42

Approach Our Two-Stage Approach

Total

1

2

3

4

5

6

7

8

9

10

* Significant difference (P \ 0.05)

Fig. 2 Superimposition of the FEM result mesh (red) on the postoperative soft tissue (grey) in a random case. Result with large error after the first FEM step (a). Accuracy improvement in the lower face after the second FEM step (b)

Fig. 1 Subdivision of facial soft tissue based on landmarks Results The results are shown in Table 1 and Fig. 2. There were statistically significant different in the simulation errors between the traditional FEM and our two-stage approaches (P \ 0.05). As hypothesized, the results of the traditional FEM simulation showed a larger amount of errors in the lower third face, mainly in the lower lip and the chin regions (zone 7–10). After the second stage of simulation was performed, the errors in zone 8–10 were statistically significant reduced. Although the errors between the two methods were relative small quantitatively, the differences shown 3D graphic views were absolutely clinically significant (Fig. 2).

Conclusion The lip and chin areas are critical in orthognathic surgery for both surgeons and patients. To achieve an accurate facial soft tissue simulation is also equally important for making a best-possible surgical plan. The results of this study confirmed our hypothesis that additional attentions in simulation should be paid in the lip and chin regions. In future study, we will significantly expend the sample size to collect more optimal combinations of force parameters for further optimization. Ultimately these optimized parameters will be selfadaptively used in forward simulation for the clinical use. We will also improve the error evaluation method. Currently the numeric (quantitative) results are not necessary reflecting the graphic (qualitative) results, as shown in Fig. 2. References [1] Soncul M, Bamber MA. (2004) Evaluation of facial soft tissue changes with optical surface scan after surgical correction of Class III deformities. Journal of Oral and Maxillofacial Surgery 62(11):1331–1340. [2] Zhang X, Tang Z, Liebschner MAK, Kim D, Shen S, Chang CM, Yuan P, Zhang G, Gateno J, Zhou X, Zhang SX, Xia JJ. (2015) An eFace-Template Method for Efficiently Generating Patient-Specific Anatomically-Detailed Facial Soft Tissue FE Models for Craniomaxillofacial Surgery Simulation. Annals of Biomedical Engineering (Epub ahead of print).

123

Int J CARS Chewing cycle counter on tablet PC using face recognition and motion tracking methods Y. Hayakawa1, M. Hirose1, B.- Y. Sun1 1 Kitami Institute of Technology, Computer Science, Kitami, Hokkaido, Japan Keywords Face recognition  Chewing cycle  Pattern matching  Optical flow Purpose The dysfunction of mastication and swallowing are caused by the growth in the mixed dentition period, the dysfunction at the post-operative period in surgical procedures, missing tooth/teeth, simply aging, etc. Also the chewing cycle period is closely related to food intake [1]. In 2012, we made the noncontact eye-blinking counter as an application of the face recognition technique and the motion tracking method. Therefore, we tried to modify the eye blinking counter and to develop a noncontact chewing cycle counter, which works on tablet PC, using image processing and recognition techniques. Methods Our eye blinking counter system employed the face pattern recognition, the eye-area detection, and the tracking using a templatematching method for the eye blink detection. The system was developed using some programmable functions in the OpenCV library. Changes between successive frames in a movie captured by a web camera (30 fps, 320 9 240 pixels) were recorded. We then applied three different methods as image processing techniques for eye blink detection: the subtraction of successive frames, eye open/closed status detection by image scanning and iris detection by a template matching method. In the chewing cycle counter, we firstly decide areas of both the nose and chin after capturing the face area, in the frontal face view in web camera. Some points (pixels) were selected in both areas to decide the transition movement between frames, due to the limitation of the calculation loading in tablet PCs. The optical flow function and others of the OpenCV library was applied to recorded the relative transition of the chin area to the nose area. Results Figure 1 shows the face-area capture and the mouth area detection. Figure 2 shows the detection of (1) food intake, (2) mastication (chewing), and (3) swallowing. We could record the chin movement during the mastication from the first bite to the swallowing. Since the movement showed attenuated wave forms, we could count the number of chewing cycles using the adequate threshold values on each wave form. Not only the counter but also the chewing function features are possibly observable to analysis the attenuation function and some irregular points on wave forms.

Fig. 2 An example of the detection of (1) food intake, (2) mastication (chewing), and (3) swallowing. The food is a piece of chocolate Due to the limitation of the calculation loading in tablet PCs, only four or five frames per second in 30 fps are calculated for drawing the attenuation curves. In different with the eye blink, the chewing is relatively slow function. Therefore the data captured at 7 or 9 fps were enough to record attenuation wave forms of the chin movement. To record the chin movement, at 30 fps, we applied the off-line analysis using a powerful PC after the data transfer from the tablet PC. Conclusion Based on our previous experience of making the eye blink counter system, we developed the system which enable to work as the chewing cycle counter. In comparison with the eye blink, the chewing cycle is not so fast phenomenon, therefore the resultant attenuated wave forms of the chin movement is possible to be recorded and analyzed in details. But, due the calculation capacity of tablet PCs, practically the off-line (not real time) analysis was carried out. Such non-contact counter and function analyzer of the chewing cycle is of value to evaluate the mastication dysfunction and the recovery. References [1] Smit HJ, Kemsley EK, Tapp HS, Henry CJ (2011) Does prolonged chewing reduce food intake? Fletcherism revisited. Appetite 57(1), 295–298, 2011.

Automatic segmentation for condylar morphometric analysis in CT and CBCT data: an in vitro validation M. Codari1,2, L. Ferreira Pinheiro Nicolielo2, J. Van Dessel2, M. Caffini3, G. Baselli4, C. Politis2, C. Sforza1, R. Jacobs2 1 Universita` degli studi di Milano, Biomedical Sciences for Heath, Milano, Italy 2 KU Leuven, Department of Imaging & Pathology, OMFS-IMPATH, Leuven, Belgium 3 Universita` degli Studi di Trento, CiMeC, Rovereto, Italy 4 Politecnico di Milano, Dipartimento di Elettronica, Informazione e Bioingegneria, Milano, Italy Keywords CBCT  MSCT  Segmentation  Condyle

Fig. 1 An example of the face-area (green square) capture and the mouth area (red rectangular) detection

123

Purpose The aim of orthognathic surgery is treatment of maxillofacial deformities to improve oral function as well as facial aesthetics with a longterm perspective. The surgery itself involves significant bone remodeling. The latter surely applies to the mandibular condyles. Condylar remodeling may remain within physiological condition or result in progressive condylar resorption [1].

Int J CARS

Diagnosis of this condition is mainly based on 3D radiological examination (Multi Slice CT (MSCT) or Cone Beam CT (CBCT)). These 3D scans enable analysis of morphological and volumetric changes during healing [2]. Since such analyses strictly depend on the accuracy of bone segmentation, it is important to have an accurate and repeatable segmentation method. For this reason, in this study we propose an automatic method to segment condyle structure in both MSCT and CBCT data. Methods The presented segmentation method combines patient adaptive thresholding and contrast enhancement techniques in order to improve the segmentation of both trabecular and cortical bone. Thresholding was done by 4 clusters (air, soft tissue, trabecular and cortical bone) k-means clustering performed on one of each two slices of the original volume. For each slice the minimum intensity value classified as cortical bone was collected. Thereafter, the global threshold was defined as the 10th percentile of the population of minimum values [3]. Then the same method was applied to the trabecular bone cluster. Secondly, to create the first segmentation mask, the image contrast was enhanced using unsharp masking technique and segmented using the cortical threshold value. Then, the trabecular bone threshold was applied to the voxels of the first mask obtaining the second segmentation mask. Once this mask was obtained, it was refined removing all the residual volumes of the segmentation process caused by noise or artifacts. Finally, it was applied to the original volume, maintaining the detail of the trabecular bone. For validation, a dry human hemimandible was scanned with 4 CBCT and 1 MSCT machine using clinical scanning protocols for condylar examination (Fig. 1). To reproduce soft-tissue attenuation, a cupper filter was used during all acquisitions. Moreover, the condyle was cutted and scanned using a microCT, which represents the gold standard for bone x-ray imaging (Fig. 1).

Fig. 1 Segmentation outcomes with different methods and modalities: SkyScan MicroCT with isotropic resolution of 0.035 mm (A), Somaton MSCT with anisotropic resolution of 0.3 9 0.3 9 0.4 mm (B), NewTom Vgi Evo CBCT with isotropic resolution of 0.15 mm (C), Accuitomo CBCT with isotropic resolution of 0.25 mm (D), Scanora CBCT with isotropic resolution of 0.25 mm (E) and Promax CBCT with isotropic resolution of 0.4 mm (C)

123

Int J CARS After image registration, all images were segmented with the proposed method and manual global thresholding. The segmented volumes, Dice similarity coefficient and percentages of bone overestimation were calculated. Signed-rank sum test was used for comparisons. Results The algorithm was tested on 6 image volumes. Qualitative comparison between the proposed algorithm and the global thresholding showed improvement of the segmentation accuracy, as it can be seen in Fig. 1. The median (IQR) segmented volume was 1515.1 mm3 (166.8 mm3) for the automatic segmentation method and 1949.9 mm3 (79.9 mm3) for the manual thresholding. Significant differences were found between segmented volume values (p \ 0.05). The median (IQR) Dice Similarity Coefficient was 0.87 (0.1), with the maximum value of 0.98 for the MicroCT image volume. Regarding the percentages of overestimation of the segmented volume compared with the MicroCT image, the median (IQR) value was 13 % (10 %) for the proposed method and 37 % (4 %) for the manual thresholding (p \ 0.05) Conclusion The proposed method represents a fully automatic alternative for condyle segmentation in both CT and CBCT data. In particular, the automatic segmentation allows to improve the quality of the trabecular bone segmentation and significantly reduce the overestimation of the segmented bone (with a median reduction of 24 % between methods), especially for high resolution images. Results are promising, nevertheless a further validation on an enlarged sample is advised. References [1] Hoppenreijs TJM, Maal T, Xi T (2013) Evaluation of Condylar Resorption Before and After Orthognathic Surgery. Semin Orthod 19:106–115. [2] Xi T, van Loon B, Fudalej P, Berge´ S, Swennen G, Maal T (2013) Validation of a novel semi-automated method for threedimensional surface rendering of condyles using cone beam computed tomography data. Int J Oral Maxillofac Surg 42:1023–9. [3] Codari M (2013) Automatic estimate of cephalometric landmarks in three-dimensional Cone Beam CT. Dissertation, Politecnico di Milano.

Computer assisted assessment of necrotic changes in mandibular bone due to osteoradionecrosis or bisphosphonate-associated necrosis

Firstly, all available CT-/CBCT data are registered on a suitable reference data set. A standard algorithm referring to normalized mutual information metrics mostly provides satisfactory results. Thereafter, the data are upsampled to identical dimensions with isotropic voxel size of 0.5 mm side length or below. In step 2, kind of envelope of the mandibular bone is segmented from the registered data covering the shape of all mandibles plus an axial rim of 2 mm soft tissue. We again segment an envelope of the spongious bone from the registered data including some rim through the cortical bone, entirely comprising the trabecular structure of all mandibles and omitting soft tissue as far as possible. Thereafter, from each data set, we generate two new image stacks. For the first one, the segmented voxels comprising the whole mandible (cortical and cancellous bone) are isolated from the registered CT data. For the second one, we cut out only the voxels of the trabecular structure. The voxels outside the segmented regions are set to a very low value, e.g. -10000. For the data sets comprising only the trabecular structure, the segmented voxels are inverted, e.g. bone with a Hounsfield number of 1500 is transformed into -1500. The new data sets are subjected to slice oriented direct volume rendering with various (mostly logarithmic) transfer functions specially designed for the respective purpose. For necrotic processes, destructive as well as sclerosing skeletal changes are observed.The first phenomenon corresponds to decreased Hounsfield values whereas sclerosation is indicated by increasing ones. Up to now, we provided 4 visualization approaches with different focus and requirements. The first two approaches require several follow ups of CT with approximately comparable acquisition parameters. The transfer functions are based on a physical color scale. By means of direct volume rendering of the voxels of the whole mandible, the first approach allows an analysis of destructive and sclerosing changes in cortical bone. For the second approach, the inverted data of the spongious bone are used for a display of trabecular loss. Both approaches are combined with a quantitative analysis, and, up to now, are limited to helical CT [1]. However, in clinical practice, the requested comparability of radiological data is often not provided. Furthermore, CBCT gains in importance. Therefore, for the first approach, the transfer function is changed to an only two-tone visualization. Thereby, changes in the cortical bone, especially destructive processes can be displayed (Fig. 1). This kind of binary visualization can be applied to helical CT as well as Cone Beam CT for the prize of loss of quantitative significance.

C. Kober1, G. Kjeller2, B.- I. Berg3 1 HAW Hamburg, Hamburg, Germany 2 The Sahlgrenska Academy, Department of Oral & Maxillofacial Surgery, Gothenburg, Sweden 3 HFZ Basel and University Hospital Basel, Mund-, Kiefer- und Gesichtschirurgie, Basel, Switzerland Keywords Mandible  Osteoradionecrosis  Bisphosphonate associated necrosis  Bone Purpose Pathological changes in mandibular bone due to oncologic treatment as radiation therapy, bisphosphonate medication, or post-zytostatic disorders are serious burdens. In diagnosis, necrotic changes, infection, or tumor relapse are to be differentiated. This abstract is about recent achievements within a detailed research project focused on CT-/CBCT-based visualization of necrotic changes Methods Since long-term cancer patients often present numerous radiological data with highly varying quality, all available data sets are considered, refraining from any limitation, but with ongoing critical view to the origin.

123

Fig. 1 Two-tone visualization of destructive processes due to progressive osteoradionecrosis in the mandibular cortical hull, arrows indicate continuing progression

Int J CARS On the other hand, besides severe destructive changes of the outer cortical shell, detailed sclerosing processes within trabecular structure are reported, e.g. in bisphosphonate or radiation necrosis (namely septic and aseptic pathologies). For this purpose, as in approach 2, we refer to the data sets with inverted trabecular voxels based on helical CT. For the logarithmic transfer function, a so called inverted temperature color scale is used (Fig. 2). As kind of control, visualization based on healthy subjects is referred to. Additionally, we compare the affected and the non-affected (or less affected) mandibular side. Up to now, this approach is restricted to helical CT.

and dark purple. Though some deterioration from 2013-11 to 2015-04 can be stated, the ipsi-/contralateral deviations are more dominant. Additionally to osteoradionecrosis, a pilot study for bisphosphonate patients was performed where progression of the described changes over the whole mandible was observed. This indicates that aseptic as well as septic changes can be captured. For differentiation of infectious processes, other radiological modalities are needed, f.i. PET. Conclusion Recent achievements for computer assisted visualization for necrotic changes in mandibular bone are presented. Besides diagnostic significance, this research is aimed at diagnosis efficiency, namely that physicians can examine the visualization at some glances. Surgical feedback is very positive for retrospective analysis as well as for the support of recent clinical cases. Ongoing work is dedicated to further evaluation of CBCT (qualitatively and quantitatively) and combined processing with PET and SPECT. Special focus is scanner independent quantitative analysis. References [1] Kober C, Kjeller G (2015) A computer assisted method for assessment of mandibular osteoradionecrosis for local screening and detection of periled regions. CARS 2015 Proceedings, Int J Comp Ass Rad Surg, 10(1) Suppl.

Effect of projection data elimination in image reconstruction of X-ray CT using algebraic reconstruction technique Y. Hayakawa1, B.- Y. Sun1, M. Hirose1 1 Kitami Institute of Technology, Computer Science, Kitami, Hokkaido, Japan Keywords X-ray CT  Projection data  Image reconstruction  ART

Fig. 2 (a–d) Visualization of sclerosing processes in trabecular bone due to progressive osteoradionecrosis, arrows indicate pathological changes, (e–f) projection of CT values to a plane through the alveolar ridge, ipsi- and contralateral side For immediate validation, we refer to detailed comparison with the original CT slices and projections of the CT data according to planes through the lesions (Fig. 2e–h). Substantial validation is provided by clinical feedback, based on retrospective analysis with known pathologies and after surgery of recent cases. Results Referring to CT and CBCT data, the two-tone visualization in Fig. 1 shows the destruction of the mandibular cortical hull due to progressive osteoradionecrosis after high dose radiation therapy. In spite of severe streak artifacts of the helical CT, geometrical changes are well displayed. Changes due to skeletal sclerosation, cannot be recognized with enough clarity. For the same patient, also sclerotic changes in trabecular bone were observed (Fig. 2a, c). Contralaterally, the alveolar ridge shows kind of uniform appearance in red and yellow color which is also observed for healthy controls. Ipsilaterally, especially besides the teeth, remarkable changes are visible, displayed in red

Purpose Colleagues in our laboratory have tried to reduce metal-induced artifacts using the projection data and the iterative restoration algorithm in not only Multi-Detectors Row CT also Cone Beam CT images for a couple of years. The comparison of the projection data between artifact-free CT slice and artifact-prone CT slice in neighbor was carried out and the reconstructed image was obtained using MLEM (Maximum Likelihood-Expectation Maximization) and OS-EM (Ordered Subsets-Expectation Maximization). Therefore, we considered the further possibility of the projection data manipulation in combination with the algebraic algorithm. Our tried iterative restoration methods are included in the category. Of course, the reduced number of projection data is directly connected to the patient’s dose reduction and the lower image quality. The purpose of the study is to explore the reduced number of projection data to be needed for maintaining adequate image quality. Methods We prepared previously used MD CT images [1]. Some meta-artifact free CT slices were selected. In the previous studies [1–3], we used 360 projection data at 1 degree intervals. Our trial is to reduce the number of projection data using the thinning-out method. The number of projection data was set at 180, 90 and 60 at 2, 4 and 6 degree intervals, respectively. Using various projection data sets, CT slices were reconstructed by ART (algebraic reconstruction technique) and the ML-EM method. The other experimental condition is to use 180 projection data at 1 degree intervals and also the number of projection data was set at 90 and 60 at 2 and 3 degree intervals, respectively. For the comparison we also carry out the image reconstruction using the FBP (Filtered back Projection) algorithm, which is the standard inverse-Fourier transfer.

123

Int J CARS Results Figure 1 shows the resultant images from three different reconstruction algorithms for the reduced projection data, such as 180 direction s in the range of 0–360 degree. The data sampling from 180 degree does not show any apparent quality changes in comparison with the data sampling from 360 degree. The reduced projection data, 180 at 2 deg. intervals and 90 at 4 deg. intervals (360 deg. in total) and 90 at 2 deg. intervals (180 deg. in total), showed no significant degradation of the image quality, but both 60 deg. at 6 deg. intervals (360 deg. in total) and 60 at 3 deg. intervals (180 deg. in total) showed the low quality in comparison with the 1 degree intervals.

Fig. 1 Resultant images from three different reconstruction algorithms for the reduced projection data, such as 180 directions in the range of 0 to 360 degree. Original image (far left), three reconstructed images by FBP, Algebra Reconstruction Technique (ART) and the Maximum Likelihood Expectation Maximization (Ml-EM) methods, respectively (from second left to far right) Conclusion The CT image reconstruction using the algebraic reconstruction technique and the ML-EM methods made possible to reduce the projection data from 1 degree to 4 degree intervals while keeping the diagnostic image quality high. This simply means the dose reduction of one-fourth. We will try the much practical simulation of the image reconstruction with the algebraic algorithm to confirm the result. References [1] Dong J, Kondo A, Abe K, Hayakawa Y (2011) Successive iterative restoration applied to streak artifact reduction in X-ray CT image of dento-alveolar region. Intl. J. of Computer Assisted Radiol & Surg 6(5):635–640. [2] Dong J, Hayakawa Y, Kannenberg S, Kober C (2013) Metalinduced streak artifact reduction using iterative reconstruction algorithms in X-ray CT image of the dento-alveolar region. Oral Surgery Oral Medicine Oral Pathology Oral Radiology, 115(2): e63–e73. [3] Dong J, Hayakawa Y, Kober C (2014) Statistical iterative reconstruction for streak artefact reduction when using multidetector CT to image the dento-alveolar structures. Dentomaxillofacial Radiology, Vol. 43, No. 5, 20130373.

Ultralow dose CT and iterative reconstruction imaging: influence on contrast noise ratio of orbital soft tissues G. Widmann1, F. Waldenberger2, D. Juranek2, P. Schullian1, A. Dennhardt2, R. Ho¨rmann3, M. Steurer1, E.- M. Gassner1, W. Puelacher2 1 Medical University of Innsbruck, Department of Radiology, Innsbruck, Austria 2 Medical University of Innsbruck, Department of CMF-Surgery, Innsbruck, Austria 3 Medical University of Innsbruck, Division of Functional and Clinical Anatomy, Innsbruck, Austria

123

Keywords Ultralow dose  Contrast noise ratio  Computed tomography  Iterative reconstructions Purpose Patients with complex sports related mid-face and orbital fractures require cranial CT during the emergency diagnostic workout and may require additional high resolution CT scans for surgical planning and computer assisted maxillofacial surgery and postoperative control. The cumulative exposition to ionizing radiation may represent a considerable risk, especially for young and middle aged patients. To reduce the radiation exposure, adherence to the ALARA principle (as low as reasonably achievable) is vital and means to urge on CT protocols which provide clinical relevant information at the lowest acceptable dose. Modern CT scanners allow for substantial dose reduction. However, the increase in noise in the traditionally filtered back projection (FBP) reconstructed low dose images may render relevant soft tissue structures such as the optical nerve and eye muscles unrecognizable. Recent application of iterative reconstruction technology such as adaptive statistical iterative reconstruction (ASIR) and model based iterative reconstruction (MBIR) may improve image quality of low dose images. The purpose of the present study was to evaluate the variability of ultralow dose protocols and iterative reconstruction techniques on contrast noise ratio of orbital soft tissues. Methods CT images of a human cadaver were obtained using high resolution protocols for surgical planning and computer assisted surgery: (a) Reference dose protocol at Computed Tomography Dose Index volume (CTDIvol) 36.69 mGy, and (b) a series of Low Dose Protocols (LDP) I-IV at CTDIvol 4.18, 2.64, 0.99, and 0.53 mGy. All images were reconstructed using FBP as the standard reconstruction method and the following iterative reconstructions: ASIR-50, ASIR-100 and MBIR. Post-processing software was used for volume segmentation of the optical nerve (ON), inferior rectus muscle (IRM) and orbital fat. Hounsfield Units (HU) and standard deviation of HU of the ON, IRM and orbital fat were measured. Contrast noise ratio (CNR) of ON and IRM was calculated. Dunn’s Multiple comparison test was used to compare each combination of protocols (a = 0.05). Analysis of Variance for dependent variables was used to test between-subject effects. Results Compared with the Reference Dose Protocol, LDP I-IV showed a dose reduction of about 88.7, 92.8, 97.3, and 98.6 %. Compared with the Reference Dose Protocol FBP, statistically significant differences of CNR for ON were shown using (a) LDP I FBP (p = 0.010), (b) LDP II FBP (p \ 0.001), ASIR 50 (p = 0.003), (c) LDP III FBP (p \ 0.001), ASIR 50 (p \ 0.001) and ASIR 100 (p = 0.001), and (d) LDP IV FBP (p \ 0.001), ASIR 50 (p \ 0.001) and ASIR 100 (p \ 0.001). For IRM statistically significant differences to the Reference Dose Protocol FBP were shown using (a) LDP II FBP (p = 0.012), (b) LDP III FBP (p \ 0.001) and ASIR 50 (p = 0.001), and (c) LDP IV FBP (p \ 0.001), ASIR 50 (p \ 0.001) and ASIR 100 (p = 0.004). Conclusion When using high resolution ultralow dose CT for surgical planning and image guided maxillofacial surgery, application of MBIR may retain diagnostic CNR of orbital soft tissues. The achieved dose levels of CTDIvol of B1 mGy may be far lower than actual reference levels. Due the substantial potential to reduce exposition to ionizing radiation from CT imaging, low dose technology and iterative reconstructions should be introduced in clinical studies on maxillofacial imaging applications.

Int J CARS Effect of ultra-low MDCT doses and ASIR and MBIR on accuracy of CAD models of the jaws A. Al-Ekrish1, S. Alfadda2, W. Ameen3, R. Ho¨rmann4, W. Puelacher5, G. Widmann6 1 King Saud University, Department of Oral Medicine and Diagnostic Sciences, Riyadh, Saudi Arabia 2 King Saud University, Department of Prosthetic Dental Sciences, Riyadh, Saudi Arabia 3 King Saud University, Advanced Manufacturing Institute, Riyadh, Saudi Arabia 4 Medical University of Innsbruck, Division of Clinical and Functional Anatomy, Innsbruck, Austria 5 Medical University of Innsbruck, Department of CMF Surgery, Innsbruck, Austria 6 Medical University of Innsbruck, Department of Radiology, Innsbruck, Austria

negative difference indicated the test model was smaller (Fig. 1). A color-coded model was also obtained which demonstrated the location, magnitude and direction of the errors according to a color-coded scale (Fig. 2). A descriptive analysis was performed to analyze the position of the errors and their severity. Suitability of the test CAD models for the production of a surgical guide for dental implant surgery was evaluated based upon the magnitude, size, and distribution of areas which showed errors larger than 0.5 mm at the following sites: crest of alveolar ridge, palate, palatal slope of ridge, and buccal slope of ridge.

Keywords CT  ASIR  MBIR  CAD Purpose Multidetector Computed Tomography (MDCT) is considered one of the most accurate modalities in computer aided design/computer aided manufacture (CAD/CAM) production of 3D models of the jaws, which is likely due to its thresholding accuracy [1]. The accuracy of 3D model production is important when MDCT images are used for implant planning with subsequent production of surgical guides for dental implant surgery. However, the increasing use of MDCT is considered one of the causes for the increasing collective dose of ionizing radiation to populations [2]. Therefore MDCT dose sparing protocols which do not adversely affect diagnostic accuracy should be used whenever possible. Reduced MDCT doses have shown significant increases in image noise and density compared with standard dose protocols. However, the iterative reconstruction techniques (IRTs) of Adaptive Statistical Iterative Reconstruction (ASIR) and Model Based Iterative Reconstruction (MBIR) have allowed improvements in the subjective quality of 3D MDCT images of the facial bones when reduced radiation doses were used, as compared with the traditionally used filtered backprojection technique (FBP) [3]. However the effect of considerable dose reductions in combination with various IRTs on the accuracy of CAD models is not known. Therefore, this study aimed to compare the surface of CAD models of the maxilla produced using ultralow doses combined with FBP, ASIR and MBIR with the surface of a reference model produced from a standard dose/FBP protocol. This information may help to clarify how the use of low dose IRT protocols may influence computer guided surgery with CAD/CAM produced surgical guides. Methods MDCT imaging of a cadaver with a completely edentulous maxilla was performed using a reference standard dose protocol reconstructed with FBP (120 kV, 100 mA, Pitch: 0.5, rotation time: 1 s., CTDIvol: 29.4 mGy), in addition to 5 low dose (LD) test protocols (kV: 80, 100; mA: 10, 35, 40; pitch: 0.5, 1; rotation time: 0.4, 0.5 s.). The CTDIvol of the LD protocols was 4.19 mGy (LD1), 2.64 mGy (LD2), 0.99 mGy (LD3), 0.53 mGy (LD4), and 0.29 mGy (LD5). Each low dose examination was reconstructed 4 times with the following techniques: FBP, ASIR 50, ASIR 100, and MBIR. A stereolithography (STL) CAD model of the maxilla was produced from each dataset using Mimics software. The STL models were imported into Geomagic software and the model from each test protocol was superimposed onto the reference model using the ‘Best Fit Alignment’ function. Differences between the test and reference models were analyzed as mean positive and mean negative differences, maximum differences, and root mean square (RMS) of the differences. A positive difference indicated the test model was larger than the reference model on the outer surface of the maxilla, and a

Fig. 1 Diagramatic representation of coronal cross-section of superimposed models demonstrating magnitude of errors as indicated by color scale; negative error marked by white arrow and positive error marked by black arrow (red and blue circles indicate the maximum errors)

Fig. 2 Color coded analysis of differences between test and reference CAD models Results Decreasing dose led to increasing magnitude and areas of errors with all the reconstruction techniques. The technique associated with the largest errors at any given dose protocol was FBP followed by, in decreasing order, ASIR 50, ASIR 100, and MBIR. The pattern of errors in FBP and ASIR seemed influenced by noise in the MDCT datasets, appearing as discrete error points coalesced together, with the amount and magnitude and distribution of the error points depending on reconstruction technique and dose. With MBIR, the pattern of error did not appear related to noise. The pattern of error seen with MBIR appeared mainly as wide homogenous areas of positive errors, with the appearance of negative errors at the lowest doses. The different pattern of error distribution precluded a direct comparison between MBIR and the other techniques. Based upon the magnitude, size, and distribution of areas of dimensional errors, CAD models from the following protocols may possibly facilitate the production of accurate and stable surgical guides: FBP/LD1; ASIR 50/LD1 and LD2; ASIR 100/LD1, LD2, and

123

Int J CARS LD3; MBIR/LD1. The following protocols have demonstrated errors mostly between 1 to 2 mm or under 1 mm but over large areas, and so their effect on surgical guide accuracy is questionable and may be better determined by analysis of the stability and accuracy of CAD/ CAM surgical guides as directly related to the jaw: FBP/LD2; MBIR/ LD2, LD3, LD4, and LD5. The following protocols demonstrated large degrees of error over large areas critical to the support and positional accuracy of surgical guides, and therefore seem to preclude the production of surgical guides with the desired clinical fit and accuracy: FBP/LD3, LD4, and LD5; ASIR 50/LD3, LD4, and LD5; ASIR 100/LD4, and LD5. Conclusion When MDCT is used for CAD of the jaws, dose reductions of 86 % may be possible with FBP, 91 % with ASIR50, and 97 % with ASIR100. The powerful effect of noise reduction with MBIR resulted in an error distribution that was different from the other techniques. Analysis of the stability and accuracy of CAD/CAM surgical guides as directly related to the jaws is needed to confirm the results obtained

123

with FBP and ASIR and to clarify the effect of MBIR and ultra-low doses. References [1] Liang X, Lambrichts I, Sun Y et al. (2010) A comparative evaluation of cone beam computed tomography (CBCT) and multi-slice CT (MSCT). Part II: On 3D model accuracy. European journal of radiology 75:270–274. [2] United Nations Scientific Committee on the Effects of Atomic Radiation. Sources and effects of ionizing radiation. in Official Records of the General Assembly, Sixty-third Session, Supplement No. 46. 2008. New York: United Nations. [3] Widmann G, Schullian P, Gassner E-M, Hoermann R, Bale R, Puelacher W (2015) Ultralow-Dose CT of the Craniofacial Bone for Navigated Surgery Using Adaptive Statistical Iterative Reconstruction and Model-Based Iterative Reconstruction: 2D and 3D Image Quality. American Journal of Roentgenology 204:563–569.

Int J CARS

Day on Integrated Patient Care Chairman: Heinz U. Lemke, PhD (D)

123

Int J CARS WatsonMD: a machine certified to be a doctor H. G. Kenngott1, M. Apitz1, S. Bader2, F. Wagner1, A. Termer1, A. Rettinger2, M. Wagner1, B. P. Mu¨ller1 1 Heidelberg University, General, Visceral and Transplantation Surgery, Heidelberg, Germany 2 Karlsruhe Institute of Technology, Institut fu¨r angewandte Informatik und Formale Beschreibungsverfahren, Karlsruhe, Germany Keywords Cognitive computing  USMLE  IBM Watson  Decisionmaking Purpose Watson is an artificially intelligent computer system developed by IBM (IBM Inc., Armonk, New York, USA) capable of answering questions posed in natural language and proved its maturity in beating humans in the television quiz show jeopardy. Since then IBM spreads the technology into every industry. In healthcare, Watson’s natural language, hypothesis generation, and evidence-based learning capabilities hypothetically allow it to function as a clinical decision support system based on trained domain specific information for use by medical professionals. In this experiment, in order to prove his capabilities, Watson was trained to pass the United States Medical Licensing Examination (USMLE) as part of the final medical exams. Methods In cooperation with IBM, a clinical decision support application using the Watson framework was used. Watson was trained on a custom medical database (corpus) that encompassed both medical textbooks and domain-specific Wikipedia articles. The used corpus contained 100 eBooks of medical literature not older than 2005, and 15,000 medical Wikipedia articles which were filtered for the categories disease, anatomy, traumatology and treatments. In order to logically connect the given information Watson had to be trained with USMLE test questions. USMLE Question online databases were selected to supply Watson with sample question-answer pairs. The Watson system was trained with 800 different question-answer pairs. Watson was then taught to format the test questions, then combined the questions with the multiple-choice answers, then compared the received text snippets and finally computed a score of the answer’s plausibility. This approach used similarity measures to compare Watson’s output paragraphs to the given multiple-choice answers or their respective derivatives. The used metrics were based on the cosine distance and on the Kolmogorov distance between topic distributions of the paragraphs obtained by Latent Dirichlet Allocation. The preliminary results of the system were generated evaluating 30 not trained USMLE test questions. Results The Watson system answered the 30 test questions with 4–8 answer options with an accuracy of 50 % right answered questions (Sensitivity Weighted Score = 0.55). Excluding the picture and figure containing questions, which currently cannot be processed by Watson, the number of right answered text based questions is 80 %. The official passing score for the USMLE step 1 is 62 %, with a mean score of 71 %. Conclusion For the first time these results showed the capability of cognitive computing getting close to passing the USMLE and the computer system being close to ‘‘graduating’’ medical school. Combining recognition algorithms for pictures and figures with the Watson System will even advance its current capability. Cognitive computing may lead to a paradigm shift on how we handle medical data and patient care in the future.

123

Noninvasive multimodal interrogation of tumor hypoxia in head and neck cancer: rationale for designing a longitudinal clinical study Y. Yu1, J. Curry2, A. Luginbuhl2, D. Cognetti2, V. Bar-Ad1 1 Thomas Jefferson University, Radiation Oncology, Philadelphia, United States 2 Thomas Jefferson University, Otolaryngology, Philadelphia, United States Keywords Hypoxia  Head and neck cancer  NIR spectroscopy  Acoustic radiation force Purpose Hypoxia in head and neck (H&N) cancer has been identified as a marker of disease aggressiveness and resistance to radiation therapy or chemotherapy. Although PET imaging is a viable technique to assess hypoxia regions, its routine or longitudinal use has many logistical and financial challenges. In this study, we aim to develop a noninvasive multimodal technique to assess hypoxia in H&N cancer inexpensively in routine clinical practice. Methods We propose to use near infrared (NIR) spectroscopy to interrogate tissue oxygenation between time points in which acoustic radiation force (ARF) from custom-designed ultrasonic transducers is energized in pulses to manipulate microvascular flow characteristics. Two 1-MHz transducers are aligned with axes intersecting at an angle and with focal spots overlapping at given depth in tissue. ARF pulses are delivered in 5 s sonication followed by 15–55 s relaxation patterns. Two different NIR spectroscopy methods are employed. In the first, two laser diodes at 685 and 830 nm wavelengths (LG-Laser Technologies, Germany) provide the NIR light source via one of 6 illumination fiber bundles, and diffusely reflected signals are collected in up to 12 collection fiber bundles. The lasers as well as signal collection channels are interlaced in time using a multi-channel optical switch (O/E Land Inc., Canada). The signals are amplified by avalanche photodiodes (Hamamatsu C5460-01, Bridgewater, NJ), and then collected using one multi-channel data acquisition card (National Instrument, Austin, TX). In the second method, commercial off-theshelf NIR spectroscopy components are used. Diffuse reflectance spectra are collected with a single 600 lm fiber, numerical aperture (N.A.) = 0.22, residing at the center of a seven 600 lm fiber probe (Ocean Optics, R600-7-VIS/NIR). The center collection fiber is connected to a 2048 pixel room temperature spectrometer (Ocean Optics, USB 2000-VIS/NIR) fitted with a grating for spectrum analysis between 200 and 1100 nm. The outer six fibers are connected to a broadband halogen light source (Ocean Optics, HL 2000). In either method, the ratio of spectral intensities at two different wavelengths is co-registered with the ARF sonication timestamps for subsequent analysis. Results The multimodal tissue microvascular interrogation technique has been validated in both in vivo preclinical and pilot clinical experiments. Mice inoculated with MCa-35 tumor cells were studied using the broadband light source and spectrometer. Diffuse reflectance intensities at 560 and 540 nm were selected for detailed analysis and correlation to immunohistochemistry. Tumor hypoxia as quantified using immunohistochemical detection of a fluorescently conjugated monoclonal antibody to the EF5 hypoxic marker was found to be strongly correlated with the spectral intensity ratio at the two wavelengths. Specifically, the difference between the endpoint values of the base trendline (see Fig. 1) was predictive of mean EF5 intensity for all vessels and for perfused vessels (p = 0.01). These findings

Int J CARS provide the first direct evidence that our multimodal spectroscopy method can be used as a noninvasive probe of tumor hypoxia.

Model-guided versus ‘‘un-precision’’ medicine: professional position of EPMA and IFCARS in Predictive, Preventive and Personalised Healthcare Olga Golubnitschaja1 and Heinz U. Lemke2 1 European Association for Predictive, Preventive and Personalised Medicine, Bonn, Germany 2 International Foundation of CARS, Ku¨ssaberg, Germany

Fig. 1 Intensity ratio I560/I540 vs. ARF sonication pulse sequence in two different tumors. Left: base trendline has no obvious difference in endpoint values; Right: rising base trendline We performed additional in vivo experiments in which mice bearing U87 glioblastoma were given VEGF antagonist for 4 days. Each animal was measured longitudinally from Day 0 (pre-therapy) to Day 4. Figure 2 shows, as an example, measurement of the mean ratio of intensities at 560 and 540 nm in one animal at one location, where the averaging was over each of 5 pulse intervals. The change in spectral signals in response to ARF sonication from Day 0 to Day 4 provides longitudinal monitoring of the effects of anti-angiogenic therapy. In select animals on select days/locations, we further performed direct (invasive) pO2 measurements using an oxygen probe. Results of these direct pO2 measurements confirmed the animal model as a valid platform for monitoring changes in tissue hypoxia/oxygenation. Thus we additionally demonstrated evidence that ARF-mediated spectroscopic signal changes can be used to monitor levels of tumor hypoxia and perfusion during anticancer therapy.

Fig. 2 Mean value of I560/I540 averaged over each of 5 ARF pulses in a longitudinal study from Day 0 (baseline) to Day 4 of administering anti-VEGF therapy in a mouse. The spectroscopic signal ratio showed daily changes consistent with vessel normalization Conclusion These in vivo preclinical and pilot clinical studies provided compelling rationale for designing a clinical study in H&N cancer patients who receive standard of care PET imaging workup for assessing tumor aggressiveness. Specifically, we envision that the multimodal interrogation technique be applied at time points from preoperative assessment, to pre-chemotherapy/radiation therapy, and weekly throughout chemoradiation therapy. ARF-mediated spectroscopy signatures will be correlated with clinical/imaging assessment, and ultimately disease control. Preclinical in vivo studies and clinical cancer diagnostic pilot study results provide compelling rationale for conducting a clinical study to evaluate the role of ARF-mediated NIR spectroscopy in longitudinal assessment of tumor hypoxia during therapeutic management of H&N cancer patients.

Keywords Predictive Preventive Personalised Medicine  Healthcare  ICT  Model-Guided Medicine  Evidence Based Medicine  Medical Information and Model Management System  Patient Specific Model Abstract Recently a position paper has been released in collaboration between EPMA and IFCARS [1] which is currently well cited by the scientific community. Why the document has attracted a particular attention worldwide? In contrast to frequently practised ‘‘un-precision’’ as well as ‘‘unpredictable, unpreventable and impersonal medicine’’ as the ‘‘global disaster in the 21st century’’ [2], the joint EPMA/IFCARS paper emphasises the consolidated position of the leading experts who are aware of the great responsibility of being on a forefront of Predictive, Preventive and Personalised Medicine. For the paradigm shift from reactive to Predictive, Preventive and Personalised Medicine, consolidated efforts at the multi-disciplinary level are essential to create a new culture in communication between individual professional domains, between doctor and patient, as well as in communication with individual social sub-groups and patient cohorts. Both societies consider long-term international partnerships and multidisciplinary projects to create PPPM relevant innovation in science, technological tools and practical implementation in healthcare. The below listed items have been nominated as the integrative approach of the long-term collaboration aiming at effective PPPM promotion. 1. Predictive, Preventive and Personalised Medicine (PPPM) as the Medicine of the Future PPPM is the advanced paradigm of medical services that enables to predict individual predisposition before onset of the disease, to provide targeted preventive measures and create personalised treatment algorithms tailored to the person. More ethical and costeffective management of health and diseases as well as the critical role of PPPM in modernisation of healthcare have been acknowledged as priorities by global and regional organizations and health-related institutions such as the Organisation of United Nations, the European Union and the National Institutes of Health [3]. 2. PPPM requires new culture of multi-disciplinary communication and multi-professional level of cooperation Epidemics of the 21st century is a number of severe chronic pathologies such as metabolic syndrome with comorbidities (about a half of billions of individuals predicted as diseased by 2030 worldwide), cannot be anymore combated by expertise and efforts of individual professional domains. However, a multi-professional level of cooperation is a big challenge, since individual professional groups are not skilled for this comprehensive task. The main issues are complex education, hybrid technologies and multi-disciplinary communication which altogether may lead to the optimal clinical decisions benefiting the patient. Considering this long-term mission, both societies EPMA and IFCARS have created the common scientific and communication platform which successfully works since 2009 [Annual EPMA/IFCARS Workshops 2009–2015]. This platform might be considered as the proof-of-principle in multi-disciplinary collaboration and trans-domain education in PPPM. 3. Model-Guided versus ‘‘un-precision’’ Medicine There are several reasons for epidemics of the 21st century and still high mortality rates in some patient cohorts such breast cancer (half of million death annually in female sub-populations) and hepatocellular carcinoma (the fifth most frequent cancer form worldwide

123

Int J CARS but the second leading cause of all cancer related deaths). These are lacks of effective screening programmes and consequently late diagnosis, multi-factorial origin of chronic diseases with cumulatively acting risk factors, patient heterogeneity, treatments developed for an averaged patient, lack of individual patient profiles and, as the consequence, frequently untargeted therapy and pathology-resistance towards currently applied treatment approaches. Therefore, an unprecise so-called ‘‘treat and wait’’ approach is inappropriate, particularly, for combating the epidemics of the 21st century. In contrast, complex individual patient profiles (family history, internal and external risk factors, monitoring of molecular profiles, ‘‘real-time’’ medical imaging, etc.) and precise disease modelling tailored to the person are expected to result in sufficiently improved medical services. The trans-domain integration of expertise of selected specialities will lead to the creation of an PPPM data-base. This data base supports a P2P cooperative environment (see Fig. 1) for creating and sharing PSMs and PMs for selected clinical domains and can be extended to additional clinical domains as experience is gained in the networking environment. P2P Best Practice PSM/PM repository Reference expert knowledge Peer Expert I

Peer Expert II Repository of workflow

Generic models and patient-spec. models

reference models (WFs, SIPs) for medical procedures, WF graph , etc.

ra WF g

Peer Expert III

etc.

Simulation and Education

ph

Peer Expert IV

P2P - Peer-to-peer PSM - Patient-specific modelling PM - Process modelling

Fig. 1 P2P ‘‘Best Practice‘‘PSM/PM representations for PPPM; the figure is taken from [1] We are proposing a comprehensive, multi-component Model Guided Medicine system, based on the visionary concept of a Medical Information and Model Management System that has the ability to achieve the desired results and can be built upon advanced ICT technology in a cost-effective manner and within a reasonable time frame. The key features of this proposed system concept (1) have been conceptualized with input from a wide community of medical practitioners; (2) have been presented and discussed at international medical meetings and associated forums; (3) have been peer-reviewed and published; (4) have been and are supported by a federal and local state government for exploratory and highly innovative projects, (5) have received expression of interest by leading industry in the medical field, and, (6) are supported by an international team whose expertise covers all of the component disciplines, including the design and management of successful large-scale ICT projects for healthcare settings. Once the information and communication technology is developed, its dissemination throughout all parts of the world will result in profound and cost-effective modernization of healthcare. The beneficiaries of these transforming methods and technologies will include patients, healthcare providers, and society at large.

123

References [1] Lemke HU, Golubnitschaja O (2014) Towards personal health care with model-guided medicine: long-term PPPM-related strategies and realisation opportunities within ‘Horizon 2020’. EPMA J, 5(1):8. [2] Andrews RJ, Quintana LM (2015) Unpredictable, unpreventable and impersonal medicine: global disaster response in the 21st century. EPMA Journal 2015, 6:2. [3] Golubnitschaja O, Costigliola V (2012) General report & recommendations in predictive, preventive and personalised medicine 2012: White Paper of the European Association for Predictive, Preventive and Personalised Medicine. EPMA J. 1(3):14.

What can we learn from EHR developments? C. Peter Waegemann1 1 Consultant on HIT, Berlin, Germany Keywords EHRs  Interoperability In 1997, the Institute of Medicine (IOM) declared electronic health record systems (EHRs) the essential technology for health systems because physicians and other practitioners routinely provided patient care without knowing what had been done previously and by whom, resulting both in wasteful duplication and in clinical decisions that did not take into account critical data related to patient health. In the 19 years since this announcement by IoM [1], EHRs have become part of the IT infrastructure in one form or other, and yet the goals of the IOM have not been achieved. The government of the United States, for instance, spent over $25 billion to support EHR implementation, but despite this large expenditure the challenge of interoperability has not been met. Also, physicians complain about EHRs’ lack of functionality, and they describe the user interface of most EHR systems as inadequate, time-consuming and cumbersome. These complaints are universal. Therefore, we must ask ourselves: What went wrong? Unfortunately, the focus of the EHR movement has been on making the paper medical record digital, rather than on medical processes and changes concerning the clinical workflow that fit e-care functionality. Also, medical hierarchies steered the leading universities and hospitals to create their own EHRs, most of which failed, to the loss of $20 + billion. Three generations of EHR systems emerged during the last thirty years. During the first generation, the goal was to capture patient information digitally. Mobility was added in the second generation to allow information access and documentation anywhere. Also, the range of functionalities has increased over time, allowing partial optimization of processes. The third generation enables new communication patterns, new medical intelligence, and the use of artificial intelligence. In the process of integrating all medical patient information into the EHR, one of the key challenges is how to arrange and organize, and in turn, access the wealth of data derived from all individuals involved in a patient’s care process, including medical and imaging devices. Often the volume of integrated data becomes too big for rapid and easy access of relevant information. We do not yet have a standardized approach for organizing patient information. Past attempts, including the CCR and CCD, have failed. As the healthcare system is changing from a system in which the care was provided in hospitals and medical offices, we see the beginning of a digital healthcare system in which the care takes place in a virtual space between the home, hospital, specialty clinics, and a range of peripheral health, wellness and fitness providers. Information

Int J CARS from a wide range of providers must be integrated and indexed according to relevance, timeliness, comorbility severity, and symptom interrelationships. In addition, Big Data in medicine will provide data from fields that have not been included in the past, such as environmental information, body sensors, nutritional analyzers, nanobots, etc. Information will not only be collected when the patient is in the medical office but also on a 24 9 7 basis, and not just through medical devices at home but also through explicit observations of daily living (ODLs), that is, timely patient observations of their symptoms and conditions. The goal of the last decade of personalized medicine through tailored medication and more personalized care plans has been influenced by the developments of the Digital Society [2], in which personalization occurs through better information flows. Conclusion Current EHR systems must be improved and prepared for interoperability. This third generation of EHR systems cannot be implemented simply by purchasing more sophisticated software. It requires thorough changes by radiologists, surgeons, other physicians, and IT professionals, and in particular a re-organization of data flows, and workflow changes. References [1] Dick RS, Steen EB, Detmer DE, The Computer-based Patient Record, National Academy Press, Washington, DC USA. [2] Waegemann CP (2012) Knowledge Capital in the Digital Society, Amazon.

Patient-specific surgical instruments: experiences, problems and prospects R. E. Ellis1 1 Queen’s University, Kingston, Canada Keywords Computer assisted surgery  Patient specific instrument  Additive manufacturing Purpose This work is a retrospective consideration of our 12 years of experience and a projection of future potential. A patient-specific instrument (PSI) is a rigid structure with two physically linked elements: an impression that matches an anatomical region, and a guide for a surgical instrument. The original concept was to process a preoperative CT scan to create the anatomical impression and to manufacture the PSI by numerically controlled machining; current practice uses CT or MRI scans and additive manufacturing to create a PSI [1, 2]. Early work showed that a PSI provides a significant decrease in technical error [1, 4]. Recent randomized clinical trials [5] cast doubt on whether there is a clinical benefit for technically mature, high volume procedures. Methods Literature was reviewed by combining general keyword searches with focused searches on major authors and surgical studies. Driving these searches was the observation that a PSI is, fundamentally, a mechanism that implements a limited but critical aspect of image guided surgery: image-to-patient registration and subsequent navigation of a drill path or cutting plane. This concept of a PSI is best illustrated by an example, which here is a case in a previously reported series [1]. The technical problem in hip resurfacing of the femur is accurate placement of a central pin, which acts as a physical guide over which cannulated reamers pass to re-shape the bone. The pin placement and component sizes were planned on a CT scan, shown in Fig. 1(A). Using custom software, a PSI was additively manufactured using a biocompatible thermoplastic; the PSI had a mating surface to the anatomy (an anatomical impression) and a hole that acted as a drill guide, shown in Fig. 1(B). This particular PSI incorporated a verification tool so that a surgeon

could contact a point outside of the anatomical target; the intra-operative application is shown in Fig. 1(C).

Fig. 1 A patient-specific instrument for hip resurfacing of the femur. (A) The resurfacing is surgically determined by the placement of a central pin (magenta cylinder) from a CT-based 3D model (white surface). (B) The instrument has a surface that is an anatomical impression, mating with the femur, and a hole that physically guides the central pin down the femoral neck. (C) The guide applied to a patient; here, the surgeon is using a verification device to ensure that the physical registration is valid Since 2005, we have performed several hundred PSI-guided cases. These have included: hip resurfacing for early-onset osteoarthritis; total hip arthroplasty; total knee arthroplasty after post-traumatic femoral malalignment; partial ankle arthroplasty; distal radius osteotomy; osteochondral autologous transplantation surgery for focal-defect repair in younger knees; hemipelvic reconstruction following sarcoma removal; peri-orbital tumor resection; and many oneof-a-kind, technically difficult, orthopedic procedures. Results The literature dates to 1991 and continues to the present day. The original proposal for PSI—to machine metal instruments for pedicle screws in spine surgery—proved to be too technically challenging and a first application was periacetabular osteotomy. Subsequent clinical applications, from 2004, focused on total knee arthroplasty. Since then there have been applications to many joints, including prospective randomized clinical trials for shoulder arthroplasty [2] and total knee arthroplasty [3]. PSI’s are widely reported to reduce intraoperative time and lead to short-term technical improvements, which are also our findings. In the longer term, it is unclear whether PSI-navigated procedures provide clinically significant improvements for high-volume procedures such as total knee arthroplasty [3]. This longer-term observation is consistent with studies on computer-assisted pedicle implantation in the spinal vertebra and computer-assisted total knee arthroplasty. We have found significant longer-term improvements for hip resurfacing, which is consistent with our 18-year successful experiences in imageguided orthopedic surgery. These observations of successes in computer-assisted procedures are carefully limited to those that are technically difficult or have highly variable outcomes. An early observation was that navigation reduces technical variance; in procedures that are not technically difficult the magnitude of such outliers is not high, and in procedures with predictable outcomes the long-term effects of such outliers is presumably tolerable. Applications of PSI to high-volume procedures, such as total hip or knee replacement, attempt to address mature and well optimized surgeries for which decades of experience is available in the literature. Accordingly, the findings of little long-term difference between PSI technology and conventional technology are unsurprising. Conclusion Patient-specific instruments are an ergonomically effective solution to simple navigation problems. PSI techniques ultimately rely on a high quality preoperative 3D image that is appropriately segmented. Osteophytes, which are often present in arthritic joints, are not well imaged in CT and MRI; this has proved problematic for selection and

123

Int J CARS verification of registration regions [4]. It is unclear whether commercial implementations of PSI adequately address this fundamental problem. In situations where a definitive physical registration cannot be performed, the PSI technique must be abandoned and the surgeon must revert to conventional technology. A recent development, reported elsewhere at this symposium [5], is to virtually link the physical registration and the instrument guidance; comparative workflows of historical, recent, and future PSI applications are illustrated in Fig. 2. If the registration is reliable and the target anatomy is nearby the registration anatomy, a virtually linked instrument is an extension of a current PSI; if registration is problematic, the virtual linkage can be used to convert the case to imageguided navigation. These extensions to patient-specific instruments bring them into a spectrum of technical solutions, potentially broadening the reach and effectiveness of them as implementations of computer-assisted surgery.

Fig. 2 Workflows for uses of a patient-specific instrument. (A) Historically, a PSI combines a registration surface and an instrument guide assembled into a single physical object (B) Recent PSI uses have a virtual linkage, from electromagnetic tracking, that increase the conceptual flexibility of a PSI. (C) In the future, a PSI could be used as registration assistance for image-guided surgery, uniting the ease of use of a PSI with a more powerful navigation system References [1] Kunz M, Xenoyannis GL, Rudan JF, Ellis RE (2010) Computerassisted hip resurfacing using individualized drill templates. J Arthroplasty 25(4):600–606. [2] Hendel, MD, Bryan JA, Barsoum WK, Rodriguez EJ, Brems JJ, Evans PJ, Iannotti JP (2012) Comparison of patient-specific instruments with standard surgical instruments in determining glenoid component position. J Bone Joint Surg Am 94(23):2167–2175. [3] Victor J, Dujardin J, Vandenneucker H, Arnout N, Bellemans J (2014) Patient-specific guides do not improve accuracy in total knee arthroplasty. Clin Orthop Related Res 472:263–271. [4] Kunz M, Balaketheeswaran S, Ellis RE, Rudan JF (2015) The influence of osteophytes depiction in CT for patient-specific guided hip resurfacing procedures. Int J Comput Assist Radiol Surg 10(6):717–726. [5] Dickinson AWL, Rasquinha BJ, Pichora DR, Ellis RE (2016) Accuracy of electromagnetic tracking and personalized guides for glenoid models. Int J Comput Assist Radiol Surg; in press.

Development of the digital patient model ‘‘laryngeal cancer’’ to support the decision-making process M. Stoehr1,2, M. A. Cypko2, H. U. Lemke2,3, A. Dietz1,2 1 University of Leipzig, Otolaryngology, Head and Neck Surgery, Leipzig, Germany

123

2

University of Leipzig, Innovation Center Computer Assisted Surgery, Leipzig, Germany 3 University of Southern California, Image Processing and Informatics Laboratory, Los Angeles, United States Keywords Digital patient model  Laryngeal cancer  Bayesian networks  Clinical decision support system Purpose Decision-making processes of patient treatment are of increasing complexity, especially in oncological disease patterns like head and neck cancer (HNC). Numerous variable factors or information entities (IEs) influence the therapeutic decision-making process (e.g. TNMstaging, operability, co-morbidities etc.). Therefore, treatment decisions are made in interdisciplinary tumor boards. Biological parameters become increasingly important and allow individualized decisions, but also contribute to an increase in complexity of treatment patterns. Modern methods of modeling information can help to support decision-making by integrating all relevant information entities (IEs) of a patient and the individual disease in a probabilistic digital patient model (DPM). At the University of Leipzig medical experts from the Department of Otolaryngology/Head and Neck Surgery together with specialists in bioinformatics from the Innovation Center Computer Assisted Surgery developed a clinical decision support system (CDSS) based on multi-entity Bayesian networks (MEBN) to support a profound and transparent decision-making [1]. A MEBN is a probabilistic graphical model which integrates the relevant IEs and their direct causal dependencies into a graph structure that are defined by conditional probabilities between directly dependent IEs. Due to its generic properties of a MEBN, the input of individual patient data results in a patient-specific Bayesian network (PSBN) which allows to determine the possibilities of yet unobserved IEs (e.g. treatment outcomes, treatment side effect, quality of life etc.). The research aims to develop a CDSS based on a previously developed concept for oncological therapy decision support. Methods The disease pattern of laryngeal carcinoma was selected as starting point for the development of a CDSS. IEs were collected from clinical practice guidelines, literature review and analysis of head and neck tumor board meetings. These IEs were integrated into the model by utilizing the existing software GeNIe 2.0 [2]. A newly developed web-tool was designed to simplify and accelerate the integration of condition probabilities into the model [3]. A validation process of the MEBN laryngeal cancer was conducted initially of the subgraph of TNM-staging, which assesses the extension of the primary tumor (T), the regional lymph node status (N), and the presence of distant metastasis (M). In this validation analysis of the subgraph TNM, n = 66 cases of laryngeal cancer were analyzed comparing the models calculation of the T, N, and M-status with the a priori given TNM-status (determined by the medical experts). The research also focused on the appropriate method and transformation of a practical and multi-functional visualization of the model. Results The treatment decision model of laryngeal cancer is designed at the University of Leipzig since 2013. Currently, the graph represented in Fig. 1 consists of over 1350 IEs connected by over 1500 dependencies. The newly developed web tool reliably transforms probability equations into natural language allowing the medical expert to quickly set the conditional probabilities. In the validation process of the subgraph TNM-staging, data of the previously analyzed laryngeal cancer cases were inserted into the graphical model from primary findings (clinical data, imaging, pathology etc.) and comprises over 324 IEs.

Int J CARS

[3]

Conference on Artificial Intelligence (AAAI-99). AAAI Press/ The MIT Press, Menlo Park, CA. 1999; 342–343. Cypko MA, Hirsch D, Koch L, Stoehr M, Strauss G, Denecke K. Web-tool to Support Medical Experts in Probabilistic Modelling Using Large Bayesian Networks With an Example of Rhinosinusitis. Stud Health Technol Inform. 2015;216:259–63.

Towards summarized treatment guidelines and studies in personalized treatment planning for complex multifactorial diseases Y. Deng1, M. Stoehr2, M. A. Cypko1 1 University of Leipzig, ICCAS, Leipzig, Germany 2 University Hospital of Leipzig, Dept. of Otolaryngology, Head and Neck Surgery, Leipzig, Germany

Fig. 1 The laryngeal cancer therapy decision model with nodes representing the patient (orange), examinations (yellow) and decisions (purple). Colored frames are added to present sub models based on different topics (listed in the column) Improvements of the model and critical reassessment of the data resulted in a correct calculation of the clinical or pathological TNM staging with increasing accuracy ([80 %) in comparison to the predetermined TNM staging. Furthermore, a visualization tool with extended functionalities was developed, which allows analyses among PSBNs to compare different cases or to focus on specific situations or special viewpoints. Conclusion The MEBN laryngeal cancer was validated in parts and is still under development, because the modeling process and its validation is very time intensive. New tools will be developed to support a more intuitive and collaborative modelling and validation, to increase the quality of models and decrease the modelling and validation time. PSBNs may support the decision process by providing increased availability and transparency of information. Personalized medicine and targeted therapy are of increasing importance in oncological treatment, so that a structured and comprehensive support is crucial for information management and decision-making. The model laryngeal cancer contributes to this trend, still demanding further expansion and subsequent validation of the model. Still, for collaborative modelling and validation, and the clinical integration of such decision-supporting tools require standardization of the models, modeling techniques, and interoperability, which should be based on the established standards (e.g. IHE, HL7, DICOM etc.). The treatment decision model of laryngeal cancer on the basis of a CDSS using MEBN may be a standard for probabilistic modeling to create sufficient PSBN analyses. The concept needs to be extended to other tumor entities to subsequently improve the treatment and outcome of patients suffering from malignancies. References [1] Stoehr M, Cypko M, Denecke K, Lemke H, Dietz A. A model of the decision-making process: therapy of laryngeal cancer. Int J CARS 2014; 9 (Suppl1): S217–S218. [2] Druzdzel MJ. SMILE: Structural Modeling, Inference, and Learning Engine and GeNIe: A development environment for graphical decision-theoretic models (Intelligent Systems Demonstration). Proceedings of the Sixteenth National

Keywords Personalized medicine  Knowledge retrieval  Guideline summarization  Patient specific modeling Purpose Clinical therapy decision making for oncological diseases is complex due to the data overflow caused by large amount of available patient data and treatment options. A collaborative meeting of a multidisciplinary clinical team (e.g. tumor board) is needed to discuss the patient situation and find the best patient-specific treatment decision. Decisions should be well-grounded based on guidelines, studies and experiences from similar patient cases. In order to support comprehensibility, transparency and reproducibility in clinical therapy decision making Lemke et al. [1] suggest the development of clinical decision support systems (CDSS) based on medical evidence using probabilistic graphical models (PGM). Besides, we suggest that a CDSS should also provide the right guidelines and studies at the right time, since the retrieval of detailed guidelines and studies is required as supplement to the PGM. Especially in rare cases or particularly complex circumstances, a personalized guideline summarization across clinical specialties is still not well supported. In this paper, we introduce an architecture for the automatic guideline and study summarization for personalized treatment planning. The index generation and guideline summarization based on patient specific model are presented. The clinical use case for the treatment of laryngeal cancer based on this architecture will be discussed. Methods The realization of the personalized guideline summarization requires two basic technologies: (1) As PGM we choose multi-entity Bayesian networks (MEBN) to compute patient specific Bayesian networks (PSBNs) [1, 2]. A MEBN represents a joint probability distribution over a number of random variables. It contains variables with multiple possible states and direct causal dependencies between these variables. Furthermore, conditional probability tables in each variable describe strength of its direct influences. By setting given patient information into a MEBN a computed PSBN represents a snapshot of specific patient situation. All states referring to observed patient information are set with values, while the values for states of unobserved entities are calculated based on the conditional probabilities. An exemplary model for the treatment of laryngeal cancer is presented in Fig. 1(f) [2]. The model includes the entities about patient status (e.g., tumor-, lymph nodes and metastases-staging, genetic factors and quality of life) and clinical interventions (e.g. therapies with corresponding risk factors).

123

Int J CARS

Fig. 2 Structure of Guideline Index

Fig. 1 System Architecture of Guidelines Summarization (2) The method of inverted index is relatively mature technologies in the field of information retrieval [3]. However, the ranking method based on patient specific Meta data is still not well studied. In this case, the prior probability and calculated probability from the PSBN will be employed to provide the patient specific context. The weighting value is: Wi;d ¼ tfi;d logðn=dfi Þ Pi tfi,d = frequency of term i in document j n = total number of documents dfi = the number of documents that contain term i Pi = the conditional probability of term i in the PSBN The ranking value of one guideline document is the sum of all the term’s weighting values. The high ranking value indicates the high relevance. The keywords with higher probability in the PSBN are boosted with higher ranking value. For example, if one keyword has a term frequency of 2, an inverse document frequency of 1.9 (10 appearances in 800 documents) and a PSBN-probability of 0.9, the W-value is 3.42. The sum value in Fig. 2(a) example is the sum of all W-values of queried terms in one document. Besides the sum value, similarity measures such as cosine similarity or structural similarity can also be applied to obtain the relevant guideline document.

123

Results Based on the aforementioned technologies, a system design for personalized guideline summarization is presented as follows: The system, illustrated in Fig. 1, consists of two main components: Frontend and backend. The clinical expert can conduct the guidelines and studies retrieval through our frontend, while our backend system can manage the index for guideline summarization based on the probabilities and dependencies from the PSBN. Backend system (1) Knowledge corpus (a)—Different types of guidelines and studies should be segmented and stored with standardized structure. This component will provide an exchangeable semantic unit for all the relevant knowledge resources referring to the target disease in the PSBN. The semantic unit guarantees that only one subject is expressed in one document. The updates of the corpus will be conducted with predefined time interval. The newly emerged medical literatures can be both crawled from MEDLINE. Some local specific experiences can be uploaded by physicians manually. (2) Knowledge indices (b)—An efficient index structure is mandatory for the knowledge retrieval. As illustrated in Fig. 2, an index based on aforementioned weighting metrics is generated. A hash tree index of keywords is created for the sake of efficiency. Besides, the equivalent terminologies in different guidelines should be normalized through data linking method. The similar concept can also be consolidated through description logic reasoning. The terms in different guidelines can thus be mapped to the keywords in the PSBN and retrieved simultaneously. Frontend system (1) Patient specific summarization (c)—The indexed guideline cannot be delivered to the end user directly. We need to compose the retrieved guideline pieces into readable text. The format of production rules will be used. The condition and outcomes are composed in ‘‘if…then…else’’ syntax. Additional explanation toward each guideline item will also be provided. (2) Graphical user interface (d)—The user can raise query, settings in this component. The PSBN provides the query suggest to facilitate the query generation, while an additional full text search is provided based on textual similarity measure. Considering an advanced laryngeal carcinoma with a clinical extension of a T4a stage with ipsilateral nodal affection and a suspicious solitary pulmonary lesion, after histological verification, a

Int J CARS summary of two different guidelines through our guideline summarization system can gather literatures for both larynx and lung cancer, so that the decision-making process in the interdisciplinary tumor boards can be better supported. Conclusion The manual guideline retrieval is labor intensive task. We have provided one possible method to realize personalized clinical guideline summarization. As a next step, the prototype will be implemented and integrated into the concept of the digital patient modeling. References [1] Lemke HU, Berliner L. Patient-specific modelling & bioinformatics in PPPM. EPMA Journal (2011) 2 (Suppl 1):181–187. [2] Cypko M.A, Stoehr M, Denecke K. (2015) Web-based Guiding of Clinical Experts through the Modelling of Therapy Decision Models using generic BN with an example of laryngeal cancer.Int J CARS,2015 10 (1), 245. [3] Kelly, D, Sugimoto (2013) CRA systematic review of interactive information retrieval evaluation studies, 1967–2006. Journal of the American Society for Information Science and Technology, 64(4), 745–770.

Integrating intelligent agents in form of Arden Syntax for computing instance related fuzziness in Patient Specific Bayesian Networks J. Gaebel1, M. Stoehr2, M. A. Cypko1 1 University Leipzig, ICCAS, Leipzig, Germany 2 University Hospital Leipzig, Dept. of Otolaryngology, Head and Neck Surgery, Leipzig, Germany Keywords Patient-specific modeling  Arden Syntax  Personalized medicine  Multi Entity Bayesian Network Purpose The aim of personalized medicine is to find the best fitting therapy for the individual patient. Because of the increasing complexity and more individualized treatment patterns in oncological diseases, a current trend is to describe the patient’s situation and the specificities in digital patient models [1]. Model based clinical decision support systems (CDSS) can assist clinicians in assessing the most suitable therapy. Cypko et al. propose therapy decision models based on Multi-Entity Bayesian Networks (MEBNs). Currently, an exemplary model for laryngeal cancer is being developed [2]. MEBNs are created as generic models, integrating first order logic (FOL) with Bayesian probability theory. The relation of nodes in a MEBN, representing medical information entities (IEs), is described by conditional probability distributions. They represent the magnitude of influence of one IE on another. Multiple instantiations of one node are defined by arguments. Depending on these arguments, their instantiation values and given patient information the model is instantiated to a patient-specific Bayesian Network (PSBN). Multiple instantiations of the same IE are needed, for example, when information changes over time (e.g. two MRIs to examine cancer growth), if family history or risk factors are considered (e.g. to estimate the likelihood of cancer), or tumor localizations are evaluated (e.g. for multiple primary tumors, lymph nodes or distant metastases). The problem of MEBNs is that instantiation values of one argument can only be ranked by their listed order, but they are not differentiated by their value itself. For example, ordered time instances cannot be distinguished based on their value; ‘‘t_1’’ is more up to date than ‘‘t_0’’ but the distance of time could be a few days or many years. Being able to differentiate instances is necessary for correct inference. Hence, instances need to be weighted individually depending on the specific argument values. We suggest solving this problem by using intelligent agents that are based on Arden Syntax.

Methods Arden Syntax, published by the HL7 international standards organization, is a programming language specially designed to represent and share medical knowledge. In Arden’s Medical Logic Modules (MLMs) single medical decisions are implemented to be used in a broader clinical context [3]. MLMs are called by clinical information systems and are optimized to be used in decision support systems. Arden’s principle of fuzziness is a tool for representing medical vagueness [4]. We applied Arden’s fuzzy sets to approach the problem with multiple instances in MEBN. When implementing single medical correlations, the use of fuzzy sets can integrate the right amount of accuracy to the actual medical fact. Or in our case it can properly depict the weighting of multiple information instances. We propose the use of Arden Syntax as intelligent agents within a MEBN-based CDSS which is adapted to the therapy imaging and model management system (TIMMS) architecture [5]. For our exemplary concept, we chose the temporal properties of the tumor characteristics (TNM-state and evaluation of different metastases). In 13 MLMs depicting the different tumor-states and clinical examinations (e.g. CT or x-ray scan), we implemented the temporal relations; meaning the more an IE dates back, the less influential it should be on the overall calculations for the assessment of the findings. We set the borders of the fuzzy sets to be the current date and the individual time point where the information deteriorates in viability (e.g. one week for the reevaluation of lung metastases under treatment, or four weeks in primary staging before treatment). The date of a specific examination is passed to the fuzzy set. Depending on its value, the applicability of the IE is set to a value between 0, meaning outdated, and 1, meaning viable. This value is then passed to the MEBN system for this specific instance. When calculating the PSBN, the information of this instance has the same (when the forwarded applicability is 1) or less effect on the result. Results Our concept presented in Fig. 1 is adapted from the concept of CDSS using MEBN proposed in [2] by integrating intelligent agents in the form of MLMs. These agents enhance the instances of IEs by adding the applicability, a value between 0 and 1. In Fig. 1, the agent (a) is listening (1) to queries and pushes (3, 4) of the MEBN (b) and TIMMS (e), and forwarding back (2) calculated specific weights if needed when computing (5) a PSBN (d). First test confirmed that the applicability correctly reduces the impact of the information on the overall results for the appropriate cases. Furthermore, MLMs can be used to send (8) messages to the user (c) through a user interface (f), when certain events occur. For example, if outdated information is detected for the reevaluation of lung metastases under treatment, the user can be made aware of that fact. Either with an alerting window or the presentation of different enhanced information instances, the user could better understand the reasoning of the system and the proposed therapy decisions.

Fig. 1 Concept of CDSS using MEBN extended to intelligent agents based on Arden Syntax

123

Int J CARS As a part of the HL7 specification Arden Syntax is an open standard. Hence, MLMs can be used to communicate between different application systems or even different institutions. In the course of IHE compliant development Arden Syntax can fit well into the designated systems. It can be seen as integral part in terms of the TIMMS architecture. Conclusion The problems that arise with instantiated information entities in MEBN can be solved with the use of intelligent agents in the form of MLMs. MLMs can integrate well into existing systems and could expand CDSSs by the needed features. However, the complexity of developing these systems and modeling the medical relations increases. The systems become more heterogeneous, which increases maintenance efforts as well. We plan to extend our Arden system to all relevant information entities and integrate it into a MEBN-based CDSS to automatically enhance the calculation. We will conduct a large trial with appropriate patient data to validate our system. With clinical experts we plan to evaluate the calculations and the reliability of the proposed decisions. References [1] Chan IS, Ginsburg GS. Personalized Medicine: Progress and Promise. Annual Review of Genomics and Human Genetics. 2011;12(1):217–44. [2] Cypko MA, Stoehr M, Denecke K, Dietz A, Lemke HU. User interaction with MEBNs for large patient-specific treatment decision models with an example for laryngeal cancer. Int J CARS. 2014;10(1). [3] Samwald M, Fehre K, de Bruin J, Adlassnig K-P. The Arden Syntax standard for clinical decision support: experiences and directions. J Biomed Inform. 2012 Aug;45(4):711–8. [4] Vetterlein T, Mandl H, Adlassnig K-P. Fuzzy Arden Syntax: A fuzzy programming language for medicine. Artif Intell Med. 2010 May;49(1):1–10. [5] H.U. Lemke, L. Berliner, Model-based patient care with a therapy imaging and model management system. Predictive diagnostics and personalized treatment: dream or reality? Nova Science Publishers (2009), 131–145.

Improving patient outcomes via bridging the radiology surgery gap R. B. Schilling1 1 EchoPixel, Inc. Mountain View, CA, United States Keywords Outcomes  Surgery  Radiology  Planning Purpose CARS, since its inception, focused on bringing radiology together with surgery. Today, CARS still remains the only effective forum for Bridging the Radiology Surgery Gap (BRSG). This paper will present some of the fundamental clinical and technical changes taking place in support of BRSG—both from radiology, as it moves towards surgical planning, and surgery as it moves towards adopting 3D visualization workstations in the operating room. Methods Effective bridging requires a common language so that the parties can fully communicate, leading to creative and synergistic thinking. Between radiology and surgery, the common language is that of visualization. An example is Interactive Virtual Reality (IVR)— consisting of a stereo display, a tracking system, a stylus, and software enabling the visualization of 3D DICOM imaging as objects in open 3D space. At RSNA 2012, a radiologist, when viewing IVR for the first time noted, ‘‘utilizing IVR will enable radiologists to speak intelligently with surgeons.’’ This expressed need from a radiologist, which was

123

subsequently supported by others, indicated that there was potentially a significant opportunity for establishing a platform of communication between radiology and surgery. Shortly thereafter, when discussing the use of workstations with a group of surgeons, it was noted that, ‘‘surgeons rarely use existing 2D workstations since they never have the occasion to open up a patient and see a 2D view.’’ The combination of a surgeon considering the use of a workstation based on the IVR platform, coupled with radiologists needing to utilize IVR to communicate with surgeons, presents an opportunity to create a common platform for both radiology and surgery to communicate effectively, thereby Bridging the Radiology Surgery Gap. Results Dr. Frandics Chan, of Stanford University, is a radiologist working on the utilization of IVR in pediatric cardiology. The challenge is to provide a vessel map for the surgeon (surgical plan) to be used as a guide during surgery. At present a piece of paper is the communication vehicle, showing a sketch of the vessels involved. In a study conducted by Dr. Chan (1) and his colleagues, they found that the use of IVR increased the sensitivity of CTA by 9 % (81 % with tomographic readout and 90 % with IVR). IVR also significantly shortened the reading time by 40 % (from 22 to 13 min). The results provided by Chan will facilitate bringing this technology to the surgical room. It is estimated that IVR could significantly reduce the time for these complex surgeries from four hours (up to 10 h) to 1.5 h. During CARS 2016 it is anticipated that the results of the surgery will be presented. According to Dr. Chan, the radiologists’ role is changing. They are becoming more involved in treatment planning by taking their understanding of the disease, retaining their interpretation, and preparing the datasets for the surgeon to visualize. This is a clear example of BRSG. Dr. Judy Yee, of UCSF, is a radiologist exploring a different way to investigate CTC data by using the IVR platform. Dr. Yee believes the technology will improve the reader’s ability to detect polyps that are often difficult to see even in traditional colonoscopy studies (2)— flat lesions that are flush against the wall of the colon and typically are less than 3 mm in height. In a benchmark test conducted by EchoPixel (3), of 48 CTC cases, that had at least one false negative polyp, IVR enabled a detection rate above 90 % for 6–9 mm polyps and more importantly, increased the detection rate of flat lesions by 20 %. During CARS 2016 the results of a clinical trial held at UCSF will be presented. Conclusion Initial results have been presented showing improvement in both Clinical Efficacy and Workflow when utilizing 3D versus 2D visualization using a platform based on Interactive Virtual Reality. The work of Hegarty, et al. (4), and Hackett, et al. (5) have provided background into the reasons to expect improvement in spatial cognitive abilities and improved performance in viewing spatial relationships when using technologies such as IVR. It is anticipated that the additional trials at Stanford and UCSF will further validate the capabilities of IVR in improving Clinical Efficacy and Workflow (patient outcomes) via Bridging the Radiology Surgery Gap. References [1] Chan, F, Aguirre, S, Bauser-Heaton, H, Hanley, F, Perry, S, Head Tracked Stereoscopic Pre-surgical Evaluation of Major Aortopulmonary Collateral Arteries in the Newborns. Radiological Society of North America 2013 Scientific Assembly and Annual Meeting, December 1—December 6, 2013, Chicago IL. http://archive.rsna.org/2013/13024673.html Accessed October 2, 2014. [2] Yee J, Novel Virtual Holography CT Colonography. International Society for Computed Tomography (ISCT) 2015.

Int J CARS

[3]

[4]

[5]

Abdomen/Pelvis and Oncology. Annual Meeting June 8, 2015 San Francisco CA. S. Aguirre, Y. Zhang. A diagnostic protocol for CT colonography interpretation using an interactive holographic system. Modelling and Visualisation Track, Computer Assisted Radiology 29th International Congress and Exhibition (CARS 2015). Barceelona Spain 2015. Hegarty M.et al. The Role of Spatial Cognition in Medicine: Applications for Selecting and Training Professionals. Charpter 11; Applied Spatial Cognition, G.L.Allen, Ed., Erlbaum, Mahwah, NJ. Hackett M. Medical Holography for Basic Anatomy Training, I/ITSEC, 2013.

In this project, a middleware industrial tool called Open Resource interface for the Network (ORiN) [2] is used as the operating room interface that comprises the core of the SCOT (Fig. 1). The function of ORiN is to link the applications that will be used and the devices that are connected to the system. As devices are made abstract, changes in devices do not alter the userside applications, and can be used as is. ORiN is a middleware tool for industrial use that was developed by the Japan Robot Association. Having all the functions needed for manipulating robots, one benefit of this system is that integration of surgical assistance robots with operating rooms and the operation of the robots is very easy. Moreover, the system has a long track record and excellent reliability, as well as flexibility as an interface. We therefore determined that ORiN is optimal to use as the base technology for our operating room interface.

Development of a prototype model of ‘‘Hyper SCOT (Smart Cyber Operating Theater)’’ J. Okamoto1, Y. Horise1, K. Masamune1, I. Hiroshi1, Y. Muragaki1 1 Tokyo Women’s Medical University, Tokyo, Japan Keywords Digital operating room  Middleware  SCOT  Networked operating room Purpose Surgical procedures have progressed dramatically with the development of various new diagnostic tools and treatment devices. Surgical navigation systems utilize 3-D positioning devices show the positions of surgical instruments on diagnostic images during procedures. Such systems are mainly used in neurosurgery, otorhinolaryngology, and orthopedic surgery, where it is both important and difficult to determine precise relationships among anatomical positions. The latest engineering developments are being successively applied to medical devices, for example, high-definition endoscopic systems and the ‘‘da Vinci’’ master-slave surgical robot, are creating a new frontier in surgery. Each diagnostic, treatment, and surgical device has thus progressed remarkably as stand-alone equipment, while operating rooms remain to solely provide a space for installing and using the equipment. There is no coordination between information from different devices or systems used in the rooms, and little progress has been made on that front. The objective of this project is to develop an operating room communication interface that enables unified online management of selected devices, synchronization of data, and recording and reorganization of data to overcome the problems of conventional operating rooms [1]. Using the interface, different types of data are collected, such as images obtained from modalities during the operation and positions of surgical implements, operative field video data, and the patient’s biological information from the surgical navigation system. Information from these data sources that is required for treatment is then sent to an application that presents the information to the surgeon and surgical staff. Methods The new Smart Cyber Operating Theater (SCOT) [1] will be comprised of an intelligent operating room for performing informationguided surgery that has greatly progressed as the basic package, with the addition of intraoperative diagnostic imaging devices and modules for each surgical department, and an online device system for integrating time synchronization data1. With the SCOT, the operating room is no longer simply a room, and integration as a single systemized medical instrument with clear functions enables low-risk, high-precision medical care that can achieve highly effective treatment.

Fig. 1 Development concept of Smart Cyber Operating Theater (SCOT) Currently we have attempted to develop SCOT that are robotized is referred to as ‘‘Hyper SCOT’’ using ORiN. Target case is assumed malignant brain tumor. Hyper SCOT consists of networked surgical equipment (surgical navigation system, intraoperative MRI, patient monitor, neurophysiological monitoring device, intraoperative flow cytometry analysis system, cautery knife, shadowless lamp), information presentation monitor collected from these equipment, further newly developed robotized operating table, surgeon cockpit and robotized video microscope. Robotic operating table is composed of a 6-DOF robot and a operating table, to achieve the patient movement for rapid intraoperative MRI imaging. Robotic video microscope can store its own position and orientation during surgery, it is possible to quickly and automatically return to the surgical operation after intraoperative MRI scanning. Results Currently, we are developing a prototype of the Hyper SCOT (Fig. 2) with Intraoperative MRI in the Institute of Advanced Biomedical Engineering & Science, Tokyo Women’s Medical University. In parallel with the development of Hyper SCOT, Robotic operating table, Robotic video microscope and Surgeon cockpit are also under development. In the operating room that integrates these devices, it is possible to automatically set the therapy environment of each case’s preset pattern. It believed to be helpful in improving the efficiency and safety of the surgery and reducing the fatigue of the surgeon. We are planning to perform the treatment of the disease animals to evaluate these systems.

123

Int J CARS

Fig. 2 Over view of Hyper SCOT prototype Conclusion In this project, we have developed a integrated operation room ‘‘Hyper SCOT’’ consists of networked surgical devices and newly robotic equipments. This project is being carried out as a collaborative Japan Agency for Medical Research and Development (AMED) project by 13 commissioned institutions for five years starting in fiscal 2014. We will strive to create a SCOT package as a new export industry of Japan. References [1] Okamoto J, Masamune K, Iseki H, Muragaki Y ‘‘Development of a next-generation operating room ‘‘Smart Cyber Operating Theater (SCOT)’’—development concept and project,’’ in CARS 2015—Computer Assisted Radiology and Surgery Proceedings of the 29th International Congress and Exhibition, Barcelona, Spain, 2015, pp. 136–138. [2] http://www.orin.jp/e/

Tablet PC as central user interface for the digital operating theatre: first results in clinical routine A. Schneider1, D. Ostler1, D. Wilhelm1,2, A. Elsherbiny Hasan1, R. Stauder1, H. Feussner1,2, M. Kranzfelder1,2 1 Klinikum r. d. Isar der TU Mu¨nchen, Research Group MITI, Muenchen, Germany 2 Klinikum r. d. Isar der TU Mu¨nchen, Department of Surgery, Munich, Germany Keywords Interface  Operating room  Tablet PC  Centralized control Purpose One big advantage of integrated operating suites is the possibility of a centralized control and status overview of all connected devices [1]. Currently this control is based mostly either on touch screen systems or wall mount control panels with additional radio controls for different devices. While the wall mount control systems require without exception an unsterile nurse for the control, the use of touch screen systems is also possible for the sterile personnel at the operating table. For this purpose the control touchscreen is covered during the preparation phase in a sterile foil. However the touchscreen can often not directly placed at the operating table in ergonomic reachability of the sterile personnel or is even hindering for the team. Voice controls, even with a respective recognition rate are mostly complex to use because of the necessary ‘‘wake-up commands’’ or repeating of commands to achieve the appropriate control [2]. These problems can be overcome by use of a handheld touchscreen (tablet) based intervention room control.

123

Methods Our integrated operating suite (Trumpf Medical Systems GmbH & Co. KG) can be completely controlled with handheld tabletPC’s (Apple iPad mini 2). The position of the handheld touchscreens is recognized continuously by the TruConnect system to first, ensure that the user is in the operating room, which means that he is not able to control the integrated devices from elsewhere without direct visual feedback. Second, with the exact localization technique based on lowenergy Bluetooth, it became possible to control with the same handheld touchscreen several operation rooms, even if these are close to each other. The control application is separated in several modules, one for each peripheral device, and gives a direct visual feedback of the current control or position. This means that the actual configuration of the correspondent device is displayed on the screen (Fig. 1). The tablet PC can be covered in a sterile foil to enable the sterile personell to control all the connected devices directly at the operating table.

Fig. 1 TabletPC user interface of integrated operating room. Pictograms of all integrated devices and control of operating table To evaluate the new handheld touchscreen control with pictograms for each function, a direct comparison with a conventional integrated operating room was done. Assessed parameters were time for execution of command and errors in execution of command (i.e. tilt of table in wrong direction). In addition, preciseness of the localisation and time for handing over of the control in two adjoined operating rooms was analyzed. Therefore data of the Bluetooth localization and logins/logoffs in the WIFI network of the different operating rooms was automatically recorded in the operating room control server and analyzed afterwards. In addition a survey of the new user interface for the integrated operating room with operating room staff regarding usability, intuitiveness and user experience was done. Results Because of the well known intuitiveness of tablet computers, all nurses reported a significant advantage regarding intuitiveness compared to other operating room systems. The Tablet PC based visual representation of the peripherals, especially for the operating table reduced the known phenomen of initial tilt in the opposite direction to 0. However, if more than one command at the same time had to be executed (i.e. at the beginning of laparoscopic operations (room lights off, operating lights off, table movement)), control in conventional operation rooms was faster than with the handheld touchscreen solution. The tablet was always correctly registered in the operating room where it was located. Switching of the internal networks between the connected operating rooms in order to control different rooms with the same tablet was possible in. It was shown in the server recorded usage protocol that the option to control both with the TabletPC control equipped integrated

Int J CARS operating rooms was used intensively during times with less personnel (night shift) and reported in the survey as very helpful. Operating theatre staff was satisfied with the tabletPC, the survey showed that pictograms simplified several tasks like control of the operating table and with sterile covering use on the mayo stand was possible without limitations. Conclusion It was shown, that Tablet PC’s are superior to other controls in reduction of faulty controls, however execution of commands needed more time compared to conventional wall panels or radio controls because of the necessity of picking the appropriate control pictogram. However, this could theoretically be overcome easily by presets integrated in the software, allowing to control multiple devices at the same time as needed usually at the beginning or end of the operation. Another advantage of the TabletPC user-interface is the option to easily integrate security functions to warn the user if i.e. operating table positions not suitable or even risky for the patient will be choosen. This can be done by a operation specific database where possible and ‘‘unallowed’’ positions are defined. To move the table in such a position, an additional confirmation is then necessary (Fig. 2).

Fig. 2 Visual warning and additional confirmation before movement of operating table to dangerous position for patient References [1] Nocco U, del Torchio S. The integrated OR Efficiency and effectiveness evaluation after two years use, a pilot study. Int J Comput Assist Radiol Surg. 2011 Mar;6(2):175–86. [2] Perrakis A, Hohenberger W, Horbach T. Integrated operation systems and voice recognition in minimally invasive surgery: comparison of two systems. Surg Endosc. 2013 Feb;27(2):575–9.

A multi-scale and multi-modality statistical model of pancreas A. Shimizu1, H. Hontani2, N. Kobayashi3, H. Shouno4, K. Mori5, C. Iwamoto6, K. Ohuchida6, M. Hashizume6 1 Tokyo University of Agriculture and Technology, Institute of Engineering, Koganei, Tokyo, Japan 2 Nagoya Institute of Technology, Nagoya, Japan

3

Saitama Medical University, Hidaka-shi, Saitama, Japan The University of Electro-Communications, Chofu, Tokyo, Japan 5 Nagoya University, Nagoya, Japan 6 Kyushu University, Fukuoka, Japan 4

Keywords Computational anatomy  CT  Microscope  Micro CT Purpose Since there must be correlation in-between medical volumes with different scales (resolutions) and modalities, such as CT, MR and microscope volumes of a patient, we could construct a statistical model between the volumes. It is, however, very challenging, because the difference in scale is much larger than those in problems solved so far and we have to bridges between not only different scales but also different modalities. The purpose of this study is to construct a multi-scale and multimodality model that bridges in-between volumes with different scales and modalities, namely CT and microscope volumes of stained cells. Once we have such model, diagnostic information in a microscope volume can be estimated from a CT volume by use of the model. This paper describes preliminary results of this study, in which the whole modeling process is not completed but several interesting results are presented. Methods The proposed modeling process is composed of (1) reconstruction of 3D microscope image, (2) registration between CT and microscope volumes, (3) modeling between the registered volumes. First, we reconstructed a 3D microscope image from a series of microscope images. Subsequently, we performed registration between the microscope and CT volumes. Third, a super-resolution algorithm for a single frame, or a single volume in this study, were developed to bridges between CT and microscope volumes. Due to space limitation of the paper, we explain step (1) and (3) below. In step (1), we construct a 3D microscopic image of a target organ by registering the series of 2D microscopic images of thin slices of the organ. The proposed method firstly extracts contours of anatomical structures, e.g. the sections of vessels, in each image and then registers the given images so that the 3D anatomical structures in a resultant 3D image have smooth surfaces. Generalized cylinder models are registered to the 3D image not only by estimating the model parameters [1] but also by non-rigidly deforming the 2D images. Step (3) is a modeling process between the registered volumes. We employed a super resolution algorithm for a single frame to link between CT and microscope volumes. This study focuses on the anchored neighborhood regression approach of the paper [2], which is one of patch based algorithms. It uses ridge regression to learn exemplar neighborhoods offline and uses these neighborhoods to precompute projections to map low resolution patches onto the high resolution domain. The advantage of the algorithm is the high computational efficiency, while keeping high performance. Results Materials used to construct the multi-scale and multi-modality model are pairs of CT and microscope images of pancreas resected from KPC mice of pancreatic cancer [3]. Micro CT volumes were also used to bridge between CT and microscope images. Figure 1 presents extracted anatomical structures used for reconstruction of the microscope volume from a sequence of 40 microscope images. As you can see smooth surfaces of three vessels are successfully reconstructed because the given 2D microscope images are appropriately registered together.

123

Int J CARS

[2]

[3]

Timofte R, Smet VD, and Gool LV : Anchored Neighborhood Regression for Fast Example-Based Super-Resolution, Proc. of IEEE International Conference on Computer Vision, 2013. Hingorani SR, Wang L, Multani AS, Combs C, Deramaudt TB, Hruban RH, Rustgi AK, Chang S, Tuveson DA : Trp53R172H and KrasG12D cooperate to promote chromosomal instability and widely metastatic pancreatic ductal adenocarcinoma in mice, Cancer Cell, 7(5):469–83, 2005.

Fully automatic definition of anatomical landmarks in medical images: a feasibility study S. Hanaoka1, Y. Nomura2, M. Nemoto2, S. Miki2, T. Yoshikawa2, N. Hayashi2, K. Ohtomo2, A. Shimizu3 1 Univ. of Tokyo Hospital, Dept. of Radiology, Tokyo, Japan 2 Univ. of Tokyo, Dept. of Computational Diagnostic Radiology and Preventive Medicine, Tokyo, Japan 3 Tokyo University of Agriculture and Technology, Graduate School of Engineering, Koganei, Japan Keywords Anatomical landmark  Nonrigid registration  Computed tomography  Automatic definition

Fig. 1 Anatomical structures extracted from a series of microscope images and used for reconstruction of the microscope volume As a pilot study, we applied the super resolution algorithm to two micro CT volumes of a pig, whose resolutions are 20 [lm] and 9 [lm], respectively. Dictionary of patches was learnt from a large number of patch pairs, or more than 300 thousand patch pairs, between low and high resolution CT volumes. Figure 2 presents an input low resolution micro CT image, the proposed super-resolution image and a high resolution image by linear interpolation. Peak signal-to-noise ratio was improved from 24.70[dB] (linear interpolation) to 27.12[dB] (proposed super-resolution). We are now extending this approach to be applicable to CT and microscope volumes.

Fig. 2 A slice image of an input low resolution micro CT volume (left), the corresponding slice image of the proposed super-resolution volume (middle) and the high resolution image by linear interpolation (right) Conclusion This paper presents preliminary results of the study for constructing a multi-scale and multi-modality statistical model that will bridges between CT and microscope volumes. In the near future, we will complete the whole process and extend the process so as to include MR, OCT and mechanical strength parameters. References [1] Mille J, Laurent DC : Deformable tree models for 2D and 3D branching structures extraction, Proc. of IEEE Conference on Computer Vision and Pattern Recognition Workshops, 149–156, 2009.

123

Purpose Anatomical landmarks in medical imaging has a wide variety of applications. For example, automatic detection of landmarks is frequently used in initialization of statistical shape model (SSM)-based segmentation process for organs [1]. However, it is very time-consuming to define anatomical landmarks manually, as well as inputting anatomical landmark positions in medical images (e.g. for machine learning). On the other hand, we can also use non-anatomically defined landmarks such as SIFT-based ones [2]. However, using SIFT-like landmarks is sometimes difficult due to its limited intermodality and inter-individual correspondence and also its limited reproducibility among datasets. Especially, SIFT-based landmark definition is hard to be used in SSM-based methods, because SSM requires a predetermined set of landmarks which are embedded into the model itself. Therefore, a new methodology will be needed in which new landmarks with anatomical background can be defined automatically. In this study, a registration-based method is presented in which new landmarks are defined based on a novel triangular consistency criterion (TCC). TCC can estimate how the target anatomical structure is determined as one point in all of the given training datasets. The proposed method is validated with 50 whole torso CT datasets and the automatically defined landmarks are illustrated. We also analyze each defined landmark and evaluate whether each landmark is defined on any anatomically meaningful structure. Methods The proposed method can basically use any arbitrary registration method. In this study, we utilized our domestic landmark-guided registration method based on demon’s algorithm [3]. This registration method can utilize both grayscale image information and the manually-inputted landmark positions. Total 197 landmark positions were inputted for total N = 50 whole-torso CT datasets. These landmarks were worked as guides for registering a couple of CT datasets precisely. All pairs of datasets were registered using manually-inputted landmarks and grayscale images. In the result, total N*(N - 1) = 2450 registration results were given. Let the mapping vector field which deforms ith dataset to fit to jth dataset be Mij(x). That means, the point x in the ith image corresponds to the point Mij(x) in the jth image. Then, the triangular consistency criterion is introduced (Fig. 1). For each triplet of datasets i, j and k, the TCC value is defined as TCCijk(x) = |x - Mki(Mjk(Mij(x))) |. That means, the TCC value

Int J CARS evaluates the inconsistency of the given three mapping vector fields. Small TCC means that the corresponding points are conserved during registration and thus we regard it as a possible landmark point detected on some anatomically prominent structure. In this study, the registration result is regarded as consistent if and only if the TCC is lesser than a threshold, d = 5 mm.

Fig. 1 The definition of the TCC. It is defined using three images and deformation fields between them. In this example, the distance between the original point x and the threefold-moved point Mki (Mjk (Mij (x))) is defined as the TCCijk (x). Note that TCCijk (y) is less than TCCijk (x) in this example, that means y is a better landmark candidate than x For one fixed dataset i, the sum of counts wherePTCC is lesser PN than d N will  be calculated  as Si ðxÞ ¼ 1=ðN  1ÞðN  2Þ j¼1;j6¼i k¼1;k6¼j;k6¼i I TCCijk ðxÞ\d . In Si(x), we searched for new landmarks by a sequential manner. Firstly, all the local maxima in Si(x) are extracted. The local maxima within d = 20 mm from all of already-defined landmarks are eliminated. This process is repeated until no residual local maxima meets the criterion. Results An exemplar result of the automatic landmark definition is shown in Fig. 2. An example of Si(x) is also shown in Fig. 2(b). Total 50 landmarks were defined. The automatically-defined landmarks include the posterior margin of bilateral kidneys, the inferior wall of center of the aortic arch, nasopharyngeal, interior sides of the bilateral femoral heads, bilateral sides of vocal cords, and so on. On the other hand, some landmarks were defined not bilaterally symmetric, which may reflect the fact that our method is not stably detect symmetric anatomical structure. Nevertheless, most of defined landmarks can be interpreted as anatomically meaningful points.

Fig. 2 (a) An example of original CT volume, a coronal crosssection. (b) The corresponding Si values. (c) Automatically defined landmarks, frontal view. (d) Lateral view

Conclusion A novel method to define landmarks from a large CT dataset was presented. In the method, stably registered positions in the given images are extracted as landmarks, using TCC criterion. Therefore, the landmark definition results is largely affected by the registration method used. Although in this study landmark-guided demon’s algorithm worked well, it will be our future work to test other registration methods. On the other hand, if the registration method used can handle different modalities (e.g. CT and MRI), our algorithm can handle mixture of datasets with multiple modalities. Therefore, it will also be our future work to test our method with multiple modalities. References [1] Heimann T, Meinzer HP (2009) Statistical shape models for 3D medical image segmentation: a review. Medical image analysis, 13(4), 543–563. [2] Lowe DG (2004) Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91–110, 2004. [3] Vercauteren T, Pennec X, Perchant A, Ayache N (2008). Symmetric log-domain diffeomorphic registration: A demonsbased approach. In Medical Image Computing and ComputerAssisted Intervention-MICCAI 2008 (pp. 754–761). Springer Berlin Heidelberg.

Texture analysis-based downscaling of pathological image for image registration with MR image T. Ohnishi1, T. Tanaka1, Y. Nakamura1, N. Nitta2, I. Aoki2, H. Haneishi1 1 Chiba University, Chiba, Japan 2 Japan Agency for Quantum and Radiological Science and Technology, Chiba, Japan Keywords Image registration  Texture analysis  Pathological image  MR image Purpose Methodologies of multi-scale analysis using several modalities of medical image have been widely developed. In order to compare the features of images at the same location, accurate image registration is required. Unlike the multi-modal image registrations for CT-MR, CTPET and so on, image resolutions between target images are extremely different such as about a few hundred times. For example, pixel sizes of the MR image obtained by 7.0T MR scanner and pathological image are about 0.20 9 0.20 mm2 and about 0.25 9 0.25 lm2, respectively. Thus, conventional metrics for multi-modal image registration such as mutual information cannot evaluate the similarity between the MRI and pathological images appropriately. If image resolutions are different, the high resolution image is scaled down to similar resolution of another image using simple averaging or Gaussian kernel. However, such a simple downscaling eliminates microscopic pattern that each organ inherently possesses. For example, in brain, white matter, gray matter and thalamic have a unique microscopic structure. The simple downscaling leads to degradation of registration accuracy. In order to enhance the each structural component in pathological image and improve the accuracy of the image registration, a texture analysis technique is introduced to downscale process. Methods Flow of the image registration involving the texture analysis-based downscaling is shown in Fig. 1. First, downscaled pathological image (DPI) is generated using texture analysis. Arbitrary window region is extracted from pathological image. The noise and high frequency contents are eliminated by Gaussian filter. Then, edge of the region is enhanced and average of edge value is defined as pixel value of DPI.

123

Int J CARS If there are many high contrast pixel values in the window region, pixel value of DPI becomes high.

Fig. 1 Flow of the image registration with downscale of the pathological image The cross-sectional MR image (CSMR) that corresponds to the DPI is extracted from the 3D-MR image. The position parameters of the plane to be extracted are determined by the image registration composed of affine registration and non-rigid registration using thin-plate spline model [1]. Similarity between the CSMR image and the pathological image is measured by the conditional mutual information (CMI) [2]. The CMI uses spatial binning for standard mutual information (MI) which means the CMI evaluates the sum of several local MI values. Position parameters are optimized by the Powell–Brent algorithm [3]. Results Image acquisition test was conducted with the whole brain resected from a pig. 3D-MR image was acquired using the 7.0T MRI system (BioSpec AVANCE-III, Bruker Biospin, Germany) with a volume coil (86 mm inner diameter, Bruker Bio-spin) for RF transmission and reception. 3D-T1 weighted images was obtained using a rapid acquisition of relaxation enhancement (RARE) sequence with a fat suppression preparation pulse (TR = 500 ms, effective TE = 24 ms, rare factor = 4, average number of scans = 2, matrix size = 450 9 450 9 256, field of view = 90 9 90 9 51.2 mm3, spatial resolution = 0.2 9 0.2 9 0.2 mm3, and scan time = 7 h 57 m 52 s). Pathological specimens were made through the formalin-fixation, block-section, paraffin-embedding and thin-slice with 5 lm thickness. Pathological specimens were stained using hematoxylin-eosin (HE) and digitalized with a virtual slide scanner (NanoZoomer 2.0-HT, Hamamatsu Photonics K.K., Japan). Image size and pixel size were about 75000 9 60000 pixel and 228 9 228 nm2, respectively. Sobel filter was used as edge detection method. Kernel radius of Gaussian filter was 8 and window size of texture analysis was set to 256 9 256 pixel. Namely, image size and pixel size after downscale were about 300 9 230 pixel and 58.4 9 58.4 lm2. Initial position parameters were manually determined. The number of the control points for non-rigid registration was set to 5 9 5 9 5. Figure 2 shows DPIs using simple averaging (DPI-SA), DPIs using texture analysis (DPI-TA) and extracted CSMRs. From comparison

123

between Fig. 2a, d, g and Fig. 2b, e, h, structural components on the DPI-TAs were clearly identified and observed than those of the DPISAs. In particular, inner patterns of third DPI-TA was dramatically enhanced. We found that all registrations were successfully conducted. Inner patterns of CSMRs were quite similar with those of DPI-TAs although slight mismatch was observed around outline of each specimen.

Fig. 2 Result of DPIs and extracted CSMRs. (a), (d), (g): DPI-SA, (b), (e), (h):DPI:TA, (c), (f), (i):CSMR Conclusion We proposed a texture analysis-based image registration scheme for pathological images and MR image. Image acquisition test was conducted with a pig brain. We confirmed that differences between structural components in pathological image were clearly enhanced and image registration between pathological images and MR image was successfully conducted. In future, we will introduce a more intelligent texture analysis method and improve the non-rigid registration method. Accuracy of registration will also be evaluated quantitatively. References [1] Fred LB. (1989) Principal Warps: Thin-Plate Splines and the Decomposition of Deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence. 11: 567–85. [2] Dirk L, Pieter S, Frederik M, Dirk V, Paul S. (2010) Nonrigid Image Registration Using Conditional Mutual Information. IEEE Transactions on Medical Imaging. 29: 19–29. [3] Press WH, Flannery BP, Teukosky SA. (2002) Numerical Recipes in C 2nd Ed., Cambridge University Press: 412–9.

Function integrated diagnostic assistance based on multidisciplinary computational anatomy: automated analysis of intramuscular fat tissue H. Fujita1, N. Kamiya2, K. Ieda1, M. Yamada1, C. Muramatsu1, X. Zhou1, T. Hara1, H. Chen3, D. Fukuoka4, H. Kato1, M. Matsuo1, T. Inuzuka1 1 Gifu University, Graduate School of Medicine, Gifu, Japan 2 Aichi Prefectural University, School of Information Science and Technology, Nagakute, Japan 3 University of Occupational and Environmental Health, Department of Anatomy, Kita-kyushu, Japan 4 Gifu University, Department of Education, Gifu, Japan

Int J CARS Keywords Multidisciplinary computational anatomy  Intramuscular fat tissue  CAD  Whole-body CT Purpose The overall research purpose in our group in the Multi-disciplinary Computational Anatomy project, which has started in Japan in 2014, is to investigate an image analysis method based on the anatomic and functional information fusion and to establish a methodology of computer-aided detection/diagnosis (CAD) systems for organ and tissue functions [1]. Specifically, our plan is to investigate on three major topics: CAD systems for functional imaging such as overall and thoracic PET/CT and SPECT/CT, CAD system for water molecular diffusion function (DWI imaging), and CAD system for articular and muscular function [2, 3]. In the special session for multi-disciplinary computational anatomy in CARS2016, we present our recent achievements based on the muscle analysis part [4]. Specifically the study subject in our talk is related to an automated analysis of intramuscular fat tissue in the lower limbs in whole-body CT images, in which we analyses the changes in intramuscular fat tissue due to muscle atrophy in the cases of amyotrophic lateral sclerosis (ALS) patients. In the ALS cases, due to the increase of adipose tissue in the muscle, CT images exhibit a decrease of gray values (CT values) in the muscle region. In this study, using the muscle modeling techniques and the symmetric property of the human body, we perform automatic recognition and analysis of intramuscular fat tissue in the skeletal muscles in the lower limbs. Methods We have previously proposed the automatic recognition method of skeletal muscles in whole body. Having a whole-body X-ray CT image as an input image, a skeletal muscle region is obtained and divided into eight regions. Next, in the lower limb region, the body axis position is determined based on the minimum difference in the areas of the left and right skeletal muscle regions. Subsequently, in the obtained cross-sectional position, the region encompassing the atrophied skeletal muscle area is automatically detected using Snakes method to achieve the recognition of the skeletal muscle and fat regions. Figure 1 shows the overview of our proposed method.

intramuscular adipose tissues by muscle atrophy in the lower limb region was targeted as an initial procedure of quantitative analysis using the gray values with the muscle model technique. In the special session for multi-disciplinary computational anatomy, we also show and discuss some recent works on other parts of model-based muscles recognition and analyses in torso CT images. Acknowledgements This work was supported in part by a JSPS Grant-in-Aid for Scientific Research on Innovative Areas (Multidisciplinary Computational Anatomy, #26108005) and a JSPS Grant-in-Aid for Young Scientists (B) (#15K21588). References [1] http://wiki.tagen-compana.org/mediawiki/index.php/Main_Page [2] Fujita H, Hara T, Zhou X, Chen H, Fukuoka D, Kamiya N, Kanematsu M, Katafuchi T, Muramatsu C, Teramoto A, Uchiyama Y (2015) A02-3 Function integrated diagnostic assistance based on multidisciplinary computational anatomy—Plan of five years and progress overview FY2014 -. Proc the First International Symposium on the Project ‘‘Multidisciplinary Computational Anatomy’’ 45–51. [3] Fujita H, Hara T, Zhou X, Azuma K, Fukuoka D, Hatanaka Y, Kamiya N, Kanematsu M, Katafuchi T, Matsubara T, Muramatsu C, Teramoto A, Uchiyama Y (2016) A02-3 Function integrated diagnostic assistance based on multidisciplinary computational anatomy models—Progress overview FY2015— Proc the Second International Symposium on the Project ‘‘Multidisciplinary Computational Anatomy’’ 91–101. [4] Kamiya N, Kato H, Zhou X, Muramatsu C, Hara T, Fujita H, Chen H (2016) Composite recognition of the iliopsoas muscle based on the muscle direction modeling in CT images. Proceedings of International Workshop on Advanced Image Technology (IWAIT 2016), paper 2A-4, 6–7.

Scale-seamless registration and visualization for microcomputational anatomy K. Mori1,2, K. Nagara2, S. Nakamura3, M. Oda2 1 Nagoya University, Information & Communications, Nagoya, Japan 2 Nagoya University, Grad School of Information Science, Nagoya, Japan 3 Nagoya University, Dept. of Respiratory Surgery, Nagoya, Japan Keywords Multi-scale registration  Micro CT  Computational anatomy  Image fusion

Fig. 1 Muscular function analysis based on the muscular model Results The proposed method was applied to 10 cases (including 5 ALS cases) for automatic recognition of the fat regions in the skeletal muscles in the lower limb. As a result, the skeletal muscles including the intramuscular fat tissue were obtained successfully by Snakes method. In the recognized muscle regions, a decrease in gray values due to an increase in fat region was confirmed. In future work, there is a need to clarify the relationship between the increase in adipose tissue in the muscle regions and advanced muscle atrophy. Conclusion The initial study of automatic analysis of intramuscular fat tissue with cases of ALS using the whole body CT images was performed and the promising result was achieved. In this study, an increase in

Purpose This paper presents scale-seamless registration for micro-computational anatomical model construction. Computational anatomical models are often utilized for medical image processing. The computational anatomical models are considered to be constructed in the following four axes: (1) the space axis, (2) the time axis, (3) the pathological axis and (4) the functional axis. Anatomical structures of human anatomy have hierarchical structures in the meanings of scale. For example, a part of the lung can be structured as the bronchi, the thin bronchi, the terminating bronchi, the alveoli and so on. The current computational anatomical model mostly targets around the anatomical structure of the lung of 1–0.5 mm. Computational anatomical models around 10 lm scales have not been actively investigated yet. Micro-focus X-ray CT devices (micro CT) can depict specimens in lm-scale image resolution. Micro CT images (or volumes) can enables us to observe micro structures of specimens. In the image processing of micro CT images, it is required to co-register micro CT images of different scales. Coarse-resolution volumes can depict global structures of a specimen, while fine resolution volumes can depict tiny structures of it. It is needed to develop coregistration method for volumes having large difference in volume

123

Int J CARS resolution (over 10 times). This paper shows a scale-seamless image registration method for volumetric images taken in different resolutions by a micro CT scanner with presenting applications of micro CT volumes of the inflated fixed lung specimen. Materials We utilized micro CT volumetric images (volumes) of inflated lung resected specimen. The procedure was approved by IRB of our institute. The lung specimens include the cases of the lung cancer. We took micro CT volumes of these specimens by a micro CT scanner (SMX-90CT Plus, Shimadzu, Kyoto, Japan) in different image resolutions of 10 and 50 lm per voxel at 90 kvP and 110 lA X-ray tube configurations. Volume reconstruction matrix size is 1024 9 1024 9 537 voxels. A voxel has isotropic resolution. Also we have utilized a micro CT volume of chicken bone purchased at a grocery store for testing the proposed scale-seamless image registration techniques. The volume resolutions of these volumes are 18um and 50um. Actual volume size is 1024 9 1024 9 548 voxels. Methods Scale-seamless image registration process consists of (a) rough and robust registration based on parameter voting and (b) fine registration based on image similarity and iterative maximization. The inputs are two volumes taken in different resolutions. A part of the lower resolution volume contains the region that are depicted by the high resolution volume. We call the high resolution volume as the floating volume. Firstly we find the initial guess of the registration parameters for registration and utilize it for the precise image registration. We also assume that the x, y and z axes are roughly co-aligned during the scanning time. Actual registration procedures are summarized as follows. (a) Pre-processing—Micro CT volume contains strong noises like ring artifacts and white noises. Median filtering is performed to reduce such noises. Then floating volume is smoothed by a Gaussian filtering to make image registration easier and then downscaled to the same resolution of the reference image. (b) Rough and robust registration based on parameter voting— Since the floating volume covers only small area of the reference volume, direct utilization of conventional image registration technique would output unstable outputs. Block-matching based image registration process is employed here to obtain the initial parameter of the precise registration process [1]. We divide the floating volume into several subblocks. Then we find the best translation and the scaling parameters for each subblock based on NCC (Normalized cross correlation). After obtaining the parameters for all subblocks, we vote these parameters into the voting space to obtain the translation and scaling parameters. (c) Fine registration based on image similarity and iterative maximization—We perform the affine registration between the down-scaled floating volume and the reference volume. The registration parameters obtained in the block-matching based image registration are utilized as the initial parameter of the affine registration. NCC metric is also utilized as similarity measure. Powell method is utilized for solving the optimization process. (d) Scale-seamless volume integration—We integrate the higher resolution volume (the floating volume) into the up-scaled lower resolution volume (the reference volume) by using the affine registration parameters obtained in the previous step. Since the intensity levels are different among these two volumes (some micro CT scans do not output a reconstructed image of intensities normalized in the H.U. unit), we adjust intensity levels to make the embedded high resolution image seamless. Histogram equalization is utilized for intensity adjustment. (e) Scale-seamless navigation among multi-resolution volume— We can visualize the integrated volume as one of scale-seamless navigation. Some areas covered by the higher resolution volume can three-dimensionally visualize the internal structure of a specimen.

123

Results Figure 1 shows some examples of our scale-seamless registration results. We can see that the higher resolution volume is embedded into the lower resolution volume. The seams of two volumes are very limited on the integrated volumes. Main structures observed on the both volumes are seamlessly connected on the integrated volume. Three dimensional visualization of the scale-seamless registered volume can visualize the rough and the fine structures (Fig. 2). Such registration process can be applied to the clinical CT scan datasets and the CT datasets of the specimen of corresponding patients. The volumetric image taken in a certain resolution does not correspond to the volume artificially generated by image upscaling or downscaling from a different resolution volume. The proposed method is quite useful for image registration of micro computational anatomical structure modeling where multi-scale volumes are handled.

Fig. 1 Scale-seamless registration results: (a) (e) (i) lower resolution volume, (b) (f) (j) higher resolution volume, (c) (g) (k) magnified view of lower resolution volume, and (d) (h) (l) registered volume. (a)–(h) are micro CT volumes of the inflated lung specimen and (i)– (l) are those of chicken wing bone

Fig. 2 Scale-seamless visualization based on scale-seamless registration results. On this image, left side is a high resolution volume and right side is a low resolution volume. Two volumes of different scales are seamlessly integrated Conclusion This paper introduce an example of scale-seamless registration of volumetric images taken in different resolutions. This basic

Int J CARS techniques will help us to understand multi-scale anatomical structure understanding from macro (mm) to micro (um) scales by using multiscale volumes. References [1] Saito F (2001) Robust Image Matching for Occlusion Using Vote by Block Matching. IEICE Trans. on Info & Sys, vol. 84-D-II, no. 10, pp. 2270–2279

Multi-disciplinary computational anatomy assisting radiology S. Kido1, N. Hashimoto1, Y. Hirano1, H. Kim2, H. Kimura 3, S. Noriki4, K. Inai4, H. Shouno5 1 Yamaguchi University, Graduate School of Science and Technology for Innovation, Ube, Japan 2 Kyushu Institute of Technology, Department of Mechanical and Control Engineering, Kitakyushu, Japan 3 Fukui University, Department of Radiology, Eiheiji-cho, Japan 4 Fukui University, Autopsy imaging centre, Eiheiji-cho, Japan 5 University of Electro-Communications, Graduate School of Information and Engineering, Chofu, Japan Keywords CAD  Sparse representation  DCNN  Autopsy imaging Purpose In the Multi-disciplinary Computational Anatomy project, computational anatomical scheme will be expanded in space-axis, time-axis, function-axis and physiological-axis. So, we expect computer-aided diagnosis (CAD) applications based on this scheme are able to support diagnosis for wide range of clinical images included not only radiological images, but also pathological images and autopsy images. Here, the purpose of our research is to develop CAD applications based on this scheme for assisting radiology. We have developed robust algorithms for analyzing pathological lungs such as diffuse lung diseases or lung nodules from the viewpoint of physiologicalaxis. And, we have developed an algorithm which estimated elapsed times after death from the viewpoint of time-axis. Methods To determine the lung regions on CT images is required for preprocessing stage of CAD algorithms for lungs. However, it is difficult to determine lung regions in cases of pathological lungs. So, we have developed the determination method of lung regions with pathological abnormalities by use of anatomical structures and lung textures. Next, we have developed and evaluated detection and classification of lung abnormalities. For lung nodules, we have developed a detection algorithm of lung nodules from temporal subtraction images based on an Ada-Boost classifier, and also developed a detection algorithm of

ground-glass opacity (GGO) from Lung Image Database Consortium (LIDC) data set based on statistical features experiment. For diffuse lung diseases, we have developed classification algorithms of opacity patterns on high-resolution CT images using a sparse representation (SR) and a deep convolutional neural network (DCNN). These CAD algorithms can classify normal and diffuse lung opacities such as consolidation, GGO, honeycombing emphysema, nodular, which are typical in diffuse lung diseases. From the view point of time-axis, we will deal ‘‘life-time images’’ those consist of living and autopsy images. So, we have developed an algorithm which can estimate elapsed times after death. In this study, robust algorithms for segmentation and registration for living and autopsy images were performed. The Haralick’s texture features and histogram features were calculated in organ regions which were extracted from postmortem CT images. The useful features to estimate post-mortem time were selected by using CFS algorithm, and post-mortem time was estimated by regression analysis using the selected features. Results In the determination of lung regions, the mean concordance rate for normal lung cases was 82.8 ± 2.4 [%], and that of pathological lung cases was 83.5 ± 1.5 [%]. Anatomical structures and lung textures are useful for determining the lung regions with pathological abnormalities. The result of detection for lung nodules was obtained from patients with lung nodules, and we got true positive rate (TPR) of 96.8 [%] with false positives (FPs) of 6.45 per case. In detection of GGO regions from patients with lung nodules in LIDC data set, we obtained TP of 82.1 [%] with 6.7 FPs per case. These results are superior to conventional methods. The classification accuracy of diffuse lung disease opacities by use of SR was 96.4 [%], and computational time for the dictionary learning was 350 [sec] and that for recognizing a VOI was 0.13 [sec]. These results are superior to conventional method. By use of DCNN, DCNN trained with natural images and CT images could classify superior compared with DCNN with CT image (91.9 [%] vs. 89.0 [%]). So, the DCNN by use of natural images are useful for classification of diffuse lung disease opacities. The averages and standard deviations of errors between the actual and the estimated post-mortem time for the two groups are 9.68 [hours] and 18.82 [hours], and 4.67 [hours] and 5.15 [hours], respectively. The results of the experiment show a possibility to estimate post-mortem time with texture features. Conclusion The CAD applications we have developed obtained from multi-disciplinary computational anatomy scheme are able to assist diagnosis for wide range of clinical images. So, these applications may be useful for assisting radiology in a variety of clinical situations.

123

Int J CARS

Poster Session Computer Assisted Radiology and Surgery— 30th International Congress and Exhibition

123

Int J CARS Evaluation of trunk deformation of scoliosis patients K. Kato1, Y. Atsuta2 1 Future University, Hakodate, Hakodate, Japan 2 Asahikawa Medical University, Asahikawa, Japan Keywords Scoliosis  Trunk deformation  Depth sensor  X-Ray Purpose Scoliosis is a disease that spine is curved sideways when viewed from the front. Most of cause is idiopathic in adolescence. The scoliosis makes deformity of thoracic progress. As a result, it has an influence on respiratory organs and the circulatory organ. Therefore, early detection of the scoliosis is important for patients. It is known to clinically that relationship of thoracic deformity and scoliosis are dynamically changed constantly by the respiratory motion. The detection systems of the respiratory motion were developed but those system detects from only still images like X-ray images. Therefore, it is difficult to obtain the change of the shape continually and could not perform an accurate evaluation. We developed the system which obtain a trunk shape continually using a depth sensor. In this paper, we present a new method to evaluate the shape change of the human trunk by the breathing. Methods This system measures stage of scoliosis progression using torsion angles of the trunk by breathing. Patients take a deep breath on their stomach, put a depth sensor behind a patient and record the shape of body trunk during a deep breathing. We used Kinect made by Microsoft Corporation as the depth sensor. The application capture the continuous depth value with 30 frames per second, and store the values. Moreover, with the depth value, we can obtain images using a camera in this depth sensor and can confirm the trunk shape visually. From these depth values, we compute the torsion angles using the ellipse approximation method. Results We evaluate the torsion angle data of scoliotic patients with the aid of the Asahikawa Medical University Hospital. The experimental participants were 2 males and a female of scoliosis aged between 10 and 16 years and 5 males of non-scoliosis aged between 18 and 24 years. A maximum torsion angle of scoliosis was observed between 0.02 and 0.06 rad. A maximum torsion angle of nonscoliosis was between 0.01 and 0.02 rad. In the t test, t-value was t(17) = 2.79 9 10-6, p \ 0.05 and these results were significantly different between scoliosis and non-scoliosis. Figure 1 shows example of graphs of the torsion angle of a lumbar part of a scoliosis patient and the non-scoliosis with the breathing movement.

Fig. 1 Torsion angle of a lumbar part with the breathing

Conclusion In this research, quantitative evaluation of scoliosis can be easily by making use of an inexpensive sensor commercially, such as the Kinect of Microsoft. Early detection of scoliosis is important for scoliosis patients. Scoliosis patients need a diagnosis system simple and inexpensive to measure quantitatively. The diagnosis of scoliosis in areas where there are no specialists will be possible through making a simple diagnosis system of scoliosis. References [1] Issei S, Yuji A (2013) ‘‘Scoliosis school screening—For efficacy and current situation—, Spine deformation medic in Hokkaido—Diagnosis and treatment—’’, Volume 54, No.2, Hokkaido Society of Orthopaedics and Traumatology. [2] Yasuhiro I, Kotaro N, Nagata T (2011) ‘‘New developments of Shiruetta screening (Hiroshima method)’’, The 42th Japan School Health and School Physician Annual Consultation.

Computational fluid dynamics and experimental evaluation of an injection suspended impeller for centrifugal blood pump L. Zhu1, X. Yang1, Y. Wu1, X. Guo1, Y. Luo1 1 Institute of Biomedical Manufacturing and Life Quality Engineering, Shanghai, China Keywords Injection suspended impeller  Hydrodynamic force  Blood pump  Numerical analysis Purpose Heart failure (HF) is one of the most important causes of morbidity and mortality in the industrialized world. It was estimated that there are more than 5.8 million HF patients in the United States and more than 23 million worldwide [1]. Left ventricular assist devices (LVADs) have been verified as an effective way for the patients who suffer from the end-stage heart failure. In order to prolong the lifespan of the LVAD, a blood pump with magnetically or hydrodynamically suspended bearing is developed to avoid the mechanical contact between the bearing and pump casing [2]. Magnetic bearings can generate high restoring forces to allow operation at relative large clearance gaps. The stability of magnetic bearing requires active control and additional sensing which result in additional energy consumption and extra space for the control system. Hydrodynamic bearings are passively suspended within the pump cavity without active control, which utilize the fundamental principle of hydrodynamic lubrication. It requires extremely small bearing gaps, generally dozens of micrometers, to generate the hydrodynamic force, resulting in increased shearing stress that is inclined to induce hemolysis. In this study, a novel injection impeller for the blood pump and a gap between casing and impeller in the large range of 0.6 mm is presented. It allows the blood pump to eliminate the sensors for active control and operate at large clearance gaps. The computational fluid dynamics (CFD) analysis was conducted to primary verify the effectiveness of injection. The experiment setup was built to measure the hydrodynamic force generated by the injection. Methods Figure 1(a) shows the current configuration of the blood pump. The pump casing with a single volute is composed of the top casing and bottom casing. The impeller is driven by the electromagnetic interaction between the stator coils within the casing and Neodymium Iron Boron magnets sealed in the impeller. In the present prototype, the impeller with twelve injection channels is proposed (Fig. 1(b)). The specific dimension of each injection outlet is 2.5 mm 9 3.5 mm, and the radial distance between center of injection outlet and the axis of impeller is 30 mm. The blood was forced to accelerate along the each injection channels and finally run out through the outlet to produce the jet flow, which induces hydrodynamic force acting on the impeller. A

123

Int J CARS

net restoring force generated in the radial and axial direction facilitates the impeller to suspend within the pump cavity without any active control.

Fig. 2 (a) Pressure distribution of the impeller with and without injection channels. (b) The experiment setup to measure the hydrodynamic force

Fig. 1 (a) Cross sectional view of the developed blood pump and schematic diagram of the pump. (b) The structure of the blood pump The pump has a diameter of 75 mm, a height of 30 mm, and diameters of the inlet and outlet both 7 mm. The mass of impeller is 50 g. The priming volume of the pump is 15 mL. The numerical analysis with CFD was performed to verify the effectiveness of injection channels. The CFD software package Fluent 15.0 (Ansys, Inc., USA) was employed to perform the numerical analysis. The parameter of working fluid was set to be same as blood, with its density to be 1055 kg/m3, viscosity to be 0.0035 Pas. The three dimensional flow was set to use the RNG k-epsilon model with enhanced wall functions turbulent. Numerical analysis was performed using model of impellers with and without injection channels. The rotation speed of impeller was set to be 1500 rpm. The pressure distribution on the two impellers would be compared to verify the effectiveness of injection. Results As shown in Fig. 2(a), the pressure distribution on the impeller with injection channels shows that the localized high pressure is existed around the injection outlet. However, there is no same phenomenon happened to the impeller without injection channels. The CFD result indicates that the injection is inclined to induce the hydrodynamic force.

123

Based on the CFD result, the test setup was built to measure the hydrodynamic force induced solely by injection channels, as shown in Fig. 2(b). The experiment comprises the following steps: 1. The force generated by the impeller without injection was measured. 2. The force generated by the impeller with injection was measured. The force was the sum of the pressure distribution. Therefore, the result of step 1 subtracted that of step 2 was the hydrodynamic force only induced by injection. The circuit was filled with a solution of 33 wt% glycerine aqueous solution of which the viscosity at room temperature is the same as 37 C blood. The blood flow rate was measured with ultrasonic flow meter (KLH-2000H, China). Pressure sensors (DMY 100, China) were applied to measure the pressure changes generated by the blood pump. The displacement was adjusted by the precision position stage (TR-101-S1, Chuo precision industrial co, Japan). Measurements were performed at the rotational speed of 1500 rpm. The force transducer (Mini40, ATI Industrial Automation Inc., Apex, NC, USA) was applied to measure the hydrodynamic force. As the rotor was axially moved up by 150 lm, a downward hydrodynamic force generated by the injection was 0.52 N. As the rotor was axially moved down by 150 lm, an upward hydrodynamic force generated by the injection was 0.62 N. The restoring force is big enough to cover its gravity force. Conclusion This paper presents a primary prototype of an injection impeller for a LVAD. The feasibility of the design is proved by the CFD, indicating that the injection impeller could induce the localized high pressure around the injection area. The hydrodynamic force solely generated

Int J CARS

by injection is measured. When the bearing gap between the impeller and casing is 150 lm, the hydrodynamic force is enough to counteract the gravity of impeller. Further optimizations will be carried out to increase the magnitude of the hydrodynamic force to improve the hydrodynamic bearing performance. References [1] Roger V L (2013) Epidemiology of heart failure. Circulation research 113(6): 646–659. [2] Takatani S (2007) Progress of rotary blood pumps: Presidential Address, International Society for Rotary Blood Pumps 2006, Leuven, Belgium. Artificial organs 31(5): 329–344.

A setup for systematic evaluation and optimization of OCT imaging in the coronary arteries S. Latus1, M. Lutz2, C. Otte1, T. Saathoff1, K. Schulz1, N. Frey2, A. Schlaefer1 1 Hamburg University of Technology, Institute of Medical Technology, Hamburg, Germany 2 Universita¨tsklinikum Schleswig–Holstein, Department of Internal Medicine III, Kiel, Germany Keywords IVOCT  Percutaneous interventions  Imaging artifacts  Phantom setup Purpose Intravascular optical coherence tomography (IVOCT) offers high spatial resolution and short image acquisition time. During percutaneous coronary interventions, IVOCT imaging is used to study the structure of the coronary arteries. Motion and deformation due to pulsation and cardiomuscular contraction lead to artifacts in IVOCT images impacting the performance of, e.g., diameter estimation [1]. To study the effect of such artifacts we have established a setup allowing for repeated in vitro experiments under controlled and realistic conditions. We describe the system setup and present reproducible results for artificial vessel phantoms. Methods Our setup provides a periodic sample motion based on the cardiac cycle [2]. We combine two water reservoirs at variable heights h1 and h2 to implement different hydrostatic pressure levels, see Fig. 1. A flexible vessel phantom is mounted between those reservoirs. By variating the heights hi the pressure conditions for the opened (po) and closed (pc) valve can be adjusted. The sample inflates or contracts depending on the pressure changes and its diameter is expected to vary proportionally, see Fig. 2.

Fig. 1 Sketch (A), and photo (B) of the experimental setup. The sample (4) is integrated between the two reservoirs of water, (1) and (6). The valve (2) is positioned at height h3. The pump (5) ensures constant water levels. The IVOCT catheter feeding (3) is connected to the sample

Fig. 2 (A) Pulsation curvature for pressure difference over time, related to adjusted valve state. The three states A, B, C, and S mark the relevant pressure values for sample motion analysis. (B) Related exemplary pulsation curvature section for setup 1. The specific pulsation states are marked For this study we used the Ilumien Optis IVOCT imaging system. We focused on static IVOCT measurements. This is in contrast to clinical practice where dynamic pullback measurements are performed that introduce an additional unknown degree of freedom. To estimate the sample diameter we segment the inner sample wall on IVOCT images. To compare the measured and expected pulsatile motion of the phantoms we validate the related diameter values of IVOCT measurements with and without pulsation. For the pulsation-less reference, we measure 120 IVOCT images over a timeframe of 6 s, segment the diameters and calculate the root mean square dRMS and the maximum standard deviation r of the set. IVOCT measurements with pulsation are measured with a fixed rate fp. We record complete pulsation curves per IVOCT measurement and calculate the dRMS and their standard deviation r for pulsation states A, B, C and S, see Fig. 2A. Motion of the sample and the catheter induce seam line artifacts in IVOCT images [1]. Seam line artifacts occur as discontinuities of the segmented walls. To quantify this artifact we introduce the seam line difference Ds, the difference in A-scan depth of the inner sample wall between the first und last allocated A-scan. Results We evaluated diameter variations for h1 = 136 cm, h2 = 96.5 cm, and h3 = 55 cm (setup I) and h1 = 118 cm, h2 = 96.5 cm, and h3 = 55 cm (setup II). The pulsation rate was set to fp = 0.6bps. We analyzed six static IVOCT measurements per height setup for pressure validation and for pulsation validation. For pressure validation we measured dRMS1 = 2.314 mm with with po1 [ po2 r1 = 0.016 mm for setup I, and dRMS2 = 2.123 mm with r2 = 0.022 mm for setup II. The diameter is proportional to the adjusted pressure. The measurements of the pulsation states A, B, C and S are rS1 = 0.024 mm, rA1 = 0.087 mm, rB1 = 0.023 mm, and rC1 = 0.030 mm for setup I, and rS2 = 0.019 mm, rA2 = 0.067 mm, rB2 = 0.013 mm, and rC2 = 0.018 mm for setup II. An exemplary diameter curvature for associated pulsation is shown in Fig. 2B. The analyzed seam line artifacts of individual pulsation states behave proportional for seam line differences and calculated diameters. Conclusion We demonstrated the reproducibility of an experimental setup to analyze and characterize the effect of pulsatile motion on IVOCT imaging. Figure 2B confirms that for the estimated pulsation curvature the specific pulsation states are reproducible with a standard deviation of below 0.09 mm. The setup is suitable for motion artifacts studies under different physiological conditions. In particular, imaging artifacts like the seam line artifact can be obtained in a reproducible way. Static behavior of IVOCT imaging is measureable for phantoms while an extension for vessels is possible. References [1] Jang IK, (2015) Cardiovascular OCT Imaging. Springer International Publishing

123

Int J CARS [2]

Weissmann NJ, Palacios IF, Weyman AE (1995) Dynamic expansion of the coronary arteries: Implications for intravascular ultrasound measurements. American Heart Journal

the other functions. Additionally, the AIC value of the quartic polynomial function was lower than that of the other functions. Therefore, we propose that the quartic polynomial function is appropriate for the regression analysis of the spectral curves.

Visualization and characterization of spectral curves using dual-energy CT for material differentiation N. Hayashi1, A. Shinozaki1, H. Takeda2, T. Ukaji2, F. Kakinuma2, Y. Fukushima3, Y. Tsushima4, A. Ogura1, T. Ogura1 1 Gunma Prefectural College of Health Sciences, Radiological Technology, Maebashi, Japan 2 Isesaki Municipal Hospital, Radiology, Isesaki, Japan 3 Gunma University Hospital, Radiology, Maebashi, Japan 4 Gunma University Graduate School of Medicine, Diagnostic Radiology and Nuclear Medicine, Maebashi, Japan Keywords Dual energy CT  Spectral curves  Differentiation  Coefficient map Purpose Dual-energy CT (DECT) systems with spectral CT technique allow the simultaneous acquisition of CT data by using two different photon spectra at 80 and 140 kVp. In contrast to single-energy CT (SECT), spectral CT by DECT allows the acquisition of not only SECT images at 140 kVp but also virtual monochromatic spectral (VMS) images at 40–140 keV, material decomposition images, and effective atomic number images. Thus, DECT transforms CT scanning from a single-parametric imaging modality into a multi-parametric one and offers novel strategies for the assessment of various diseases. In addition to being less susceptible to beam-hardening and metal streak artifacts, VMS images also provide more accurate and reproducible attenuation measurement. Iodine metric analysis by spectral CT has been demonstrated to be superior to SECT in the diagnosis and differentiation of various pathologies including hepatic, lung, and thyroid tumors. Recently, VMS imaging has been found to have multiple clinical applications in diagnosing pulmonary embolism, staging gastric cancer, and differentiating small hepatocellular carcinoma from small hepatic hemangioma; the VMS images in these cases were visually and subjectively evaluated by radiologists using spectral curves. These inherent characteristics of spectral CT might have potential clinical applications. For the quantitative analysis of VMS images, it is necessary to identify the quantitative character of the spectral curves and visualize the character map. The aim of this study was to develop an optimal procedure for the visualization and characterization of spectral curves using DECT for material differentiation. Methods We hypothesized that the coefficient of a regression function of the spectral curves is a quantitative character. For the visualization and characterization of the spectral curves, we developed coefficient maps representing the quantitative characters of the spectral curves. The optimization procedure consisted of the following steps: (1) obtaining the DECT images (tube voltage, Sn140/100 kVp; tube current, 374/289 mA; CTDIvol, 14.4/ 9.8 mGy; pitch, 0.6; rotation time, 0.5 s); (2) obtaining the spectral curves of each pixel of the DECT images (effective energy, 40–190 keV); (3) determination of the optimized regression functions (linear, quadratic, cubic, quartic, quintic, sextic, septic, exponential, corrected exponential, and biexponential functions) using coefficients of determination and Akaike’s Information Criterion (AIC); and (4) mapping of each order coefficient of the regression function. To evaluate the efficacy of the newly developed procedure, we performed a phantom imaging study to evaluate its ability to differentiate four different tissues (breast, brain, liver, and adipose). Results Figure 1 shows the results of the coefficient determination for each regression function. The R2 values of the quartic, quintic, sextic, and septic polynomial functions were significantly higher than those of

123

Fig. 1 Results of the determination of the optimized regression functions. Figure shows coefficients of determination for each regression function Figure 2 shows an original CT image (140 kVp), a first order coefficient map of the quartic polynomial function, and a VMS image (100 keV). The first order coefficient map and the VMS image indicate the quantitative character of the spectral curves.

Fig. 2 Results of the character maps of the spectral curves of each pixel. Figure shows an original CT image (140 kVp), a first-order coefficient map of the quartic polynomial function, and a virtual monochromatic spectral (VMS) image (100 keV) Conclusion Since mass attenuation is shown as a function of photon energy, each pure substance has a unique spectral attenuation curve. The X-ray attenuation of various tissues can be expressed by a pair of known pure substances. Based on these principles, spectral CT imaging by DECT allows the accurate quantitative and material decomposition analysis of regions of interest (ROIs). Previous studies have demonstrated the applicability of the slope of the spectral curve for the differential diagnosis of benign and malignant thyroid nodules. However, for the analysis of the spectral curve, it is necessary to set the ROIs on the lesions. In this study, we found that the first order coefficient of the regression of the quartic polynomial function and the VMS image reveal the quantitative character of the spectral curves. Additionally, we performed the visualization of the coefficient maps. It might be possible to visually differentiate various materials and lesions using characterized maps.

Cone-beam CT for point-of-care detection of acute intracranial hemorrhage J. Xu1, H. Dang1, A. Sisniega1, W. Zbijewski1, J. W. Stayman1, X. Wang2, D. Foos2, N. Aygun3, V. Koliatsos4, J. Siewerdsen1,3

Int J CARS 1

Johns Hopkins University, Biomedical Engineering, Baltimore, United States 2 Carestream Health, Rochester, United States 3 Johns Hopkins University, Radiology, Baltimore, United States 4 Johns Hopkins University, Neurology, Baltimore, United States Keywords Cone beam CT  Point of care imaging  Neuroradiology  New system design Purpose Imaging of intracranial hemorrhage (ICH) is important for diagnosis and treatment of stroke and traumatic brain injury, as well as for monitoring post-operative patients in the ICU. While CT and MRI are mainstays for detection and monitoring of ICH, high-quality imaging at the point of care in application areas such as the ICU could improve time to diagnosis and reduce adverse events associated with patient transport [1]. Cone-beam CT (CBCT) systems are well-suited to such point-of-care application, typically featuring small footprints and lower cost, but have conventionally been challenged by noise and artifacts for low-contrast, soft-tissue imaging, such as acute ICH (40–80 HU contrast, *1 mm spatial resolution). This work describes the development and characterization of a dedicated cone-beam CT (CBCT) system for point-of-care ICH imaging. Methods System Design and Development. A cascaded systems image quality model [2] was used to compute the system spatial resolution (modulation transfer function, MTF) and noise (noise-power spectrum, NPS) as a function of system configuration and artifact corrections [3]. The task-based detectability index, d0 , was computed from the MTF, NPS, and a low-contrast (*40 HU), midfrequency ICH detection task as a figure of merit for system design. The prototype CBCT head scanner shown in Fig. 1(a) was developed using the image quality model as a guide to optimal system geometry, source and detector configuration, and acquisition/reconstruction techniques. The system is now in translation to first clinical studies in the neurological ICU.

Bowtie+ High Gain

100 0

-100 -200 Fig. 1 The prototype CBCT head scanner under development for point-of-care application in the ICU. Imaging performance in phantom suggests good low-contrast visibility at low dose. Use of HG readout combined with a bowtie filter increases CNR/dose

Prototype CBCT Imaging Performance. The performance of the prototype was assessed as a function of acquisition technique (kVp and dose), detector readout gain mode, and the implementation of various bowtie filters. Use of high gain (HG) readout is beneficial in terms of additive electronic noise but reduces the exposure range over which the detector may operate without saturation. Use of a bowtie filter yields a more uniform exposure to the detector, reduces dose and x-ray scatter, and allows HG readout without detector saturation. Two experiments were performed using an anthropomorphic head phantom containing ICH-simulating spheres, with dose (D0) measured at the center of a 16 cm diameter CTDI phantom at 90 kVp over 720 projections. First, the performance in LG detector mode (saturation exposure = 0.8 mAs/pulse) was characterized at D0 = 18 mGy. Second, performance in HG detector mode (saturation exposure = 0.1 mAs/pulse without a bowtie) was characterized at D0 = 7 mGy using a custom-designed Al bowtie with moderate curvature and thickness. Scatter and beam hardening corrections were applied as in [3], and the image was reconstructed using filtered back projection (FBP). The contrast, noise, and contrast-to-noise ratio (CNR) were measured in the ICH-simulating sphere within the head phantom. Model Based Reconstruction: Shift-Variant Penalties for Optimal Detectability. A model-based reconstruction framework has been implemented for the prototype head scanner, based on penalizedweighted least squares (PWLS) reconstruction modified to include a shift-variant penalty strength to maximize ICH detectability. Conventional PWLS minimizes the statistically weighted difference between the image estimate (Al) and the measured line integrals (l) in the projection domain weighted by the data certainty (W). Additionally, an image roughness penalty (|Wl|p) enforces smoothness between neighboring voxels: ^ u ¼ arg min jAl  lj2 þ bjWljp l

The magnitude of regularization applied to the image is controlled by the parameter b, which is traditionally a shift-invariant (scalar) value, with higher b and/or lower data certainty providing increased image smoothing. Previous work showed how a shift-variant penalty can enforce uniform spatial resolution [4] or noise throughout the image. In this work, we developed a method for spatially varying penalty strength (b map) that maximizes the local detectability index at each location in the image. Reconstructions were performed using the modified PWLS method and a digital head phantom, with simulated Poisson noise, I0 = 2 9 105 photons/pixel, 720 projections over 360, and 0.5 mm isotropic voxels. Results Figure 1(a) shows the prototype head scanner, featuring a 43 9 43 cm2 flat-panel detector (FPD) with 30 frames per second readout rate, a 15 kW rotating anode x-ray source with 0.6 focal spot size, and a relatively compact, upright U-arm geometry with source– axis–distance = 550 mm and source–detector–distance = 1000 mm. System design was guided by the task-based image quality model in Eq. 1. As shown in Fig. 1(b), simulated ICH is well seen in an anthropomorphic head phantom, with operation in HG detector mode and addition of a bowtie filter provide a *47 % improvement in CNR per unit dose. Figure 2(a) shows the spatially varying b map values in an axial slice near the skull base. The region posterior to the skull base (location 1 in Fig. 2b) has lower statistical certainty (longer line integrals and higher noise) and experiences strong smoothing in traditional PWLS. The spatially varying b map reduces smoothing in regions with lower data certainty and improves the spatial resolution and detectability for small bleeds. Figure 2(b) also shows an image reconstruction with the spatially varying b map, demonstrating that relatively small (2 mm diameter), low-contrast (50 HU) lesions exhibit excellent visibility and increased the detectability index over a

123

Int J CARS conventional PWLS reconstruction in location 1, where there was oversmoothing due to the data certainty weights.

Spaally varying β for opmal d’ Spaally Convenonal Varying β

1

1

2 2 Fig. 2 An image reconstructed with optimal d’ b maps and conventional PWLS (inset only). Simulated mm-scale bleeds exhibit excellent visualization, with a spatially varying b map improving the spatial resolution and detectability in location 1 Conclusion The development of a CBCT scanner prototype dedicated to point-ofcare head scanning was reported, and imaging performance with advanced model-based reconstruction techniques is consistent with reliable visualization of small, low-contrast acute ICH. The scanner prototype is being deployed in first clinical studies in the NICU at our institution in 2016. References [1] Gunnarsson T, Theodorsson A, Karlsson P, et al. (2000) Mobile computerized tomography scanning in the neurosurgery intensive care unit: increase in patient safety and reduction of staff workload. J Neurosurg 93:432–436. doi: 10.3171/jns.2000.93.3.0432 [2] Siewerdsen JH, Antonuk LE, El-Mohri Y, et al. (1998) Signal, noise power spectrum, and detective quantum efficiency of indirect-detection flat-panel imagers for diagnostic radiology. Med Phys 25:614. [3] Sisniega A, Zbijewski W, Xu J, et al. (2015) High-fidelity artifact correction for cone-beam CT imaging of the brain. Phys Med Biol 60:1415–39. doi: 10.1088/0031-9155/60/4/1415 [4] Cho JH, Fessler JA (2015) Regularization Designs for Uniform Spatial Resolution and Noise Properties in Statistical Image Reconstruction for 3-D X-ray CT. IEEE Trans Med Imaging 34:678–689. doi: 10.1109/TMI.2014.2365179

Design and implementation of a web based medical image viewer architecture R. Ellerweg1, D. Reuter1, E. Sta¨rk1, P. Weir2 1 Fraunhofer, Institut fu¨r Angewandte Informationstechnik FIT, Sankt Augustin, Germany 2 NUMA Engineering Services Ltd, Dundalk, Ireland

123

Keywords Web technologies  Medical viewer  NIFTI/VTK  Image processing Purpose In current web-based DICOM viewer implementations, the image processing component is either located on the client or on the server. Client-side solutions such as the X Toolkit [1] excel in their graphical capabilities and performance, but since the image data needs to be present on each client, collaboration scenarios are difficult to implement. Server-side solutions such as Orthanc [2] overcome this issue because the image processing component can access the image data very quickly. But on the other hand, their graphical capabilities are typically rather restricted compared to client-side solutions. For example, most server side implementations show the axial slice only. In this paper we present a service architecture which provides quick and centralized access to the image data as well as enhanced image processing capabilities. This architecture is used in the GoSmart environment [3] which is a planning tool for minimal invasive cancer treatments. Methods The service architecture is divided into three logical layers. Layer one handles the effective storage of the medical images. Layer two connects the different components with appropriate protocols and is responsible for the image processing. Finally layer three defines how the generated images appear on the front end. Figure 1 shows the layers, their components and the communication among those components.

Fig. 1 The layered architecture of the medical image service Storage Layer: On the one hand, the storage layer needs to handle a huge amount of data effectively, on the other hand it needs to provide quick access. To fulfil these requirements we chose a two level storage, containing a permanent storage which is scalable (e.g.: a cloud service) and a cache for quick data access (e.g.: the hard disc of a particular server). Storage requests always look in the cache first. Only if data is not available in the cache, is it downloaded from the permanent storage. To allow developers to integrate a different storage mechanism, the whole storage layer is implemented following a reflection based plugin concept. Medical Image Service Layer: The purpose of layer two is the connection of the components, as well as the processing of the medical

Int J CARS images. The connection between client and storage is done by a web API. Through this web API, volumes (NIFTI files) and meshes (VTKs) are published under a unique identifier. Once a volume is available in the storage layer, all further requests are handled by the viewer hub. For instance, if a client needs an axial slice from a volume with a particular contrast window, it would send a request to the viewer hub. Obviously, a sufficient connection speed between client and viewer is crucial for allowing high performance GUIs. To address this issue, the hub negotiates the best protocol possible, starting from HTML5 web sockets, and dropping back to AJAX long polling if this is not available. The image processing toolkit is based on two well established libraries, namely SimpleITK and VTK. SimpleITK provides basic filter operations which are used to generate slices for the basic anatomical plane (sagittal, coronal, axial) and their settings (contrast, position etc.). VTK is used to outline anatomical structures, e.g.: an organ in the slice. Client Layer: The previous layer defined the interface of the service architecture, which can be called via native web technologies (HTML/CSS/JS) already. However, this approach is difficult since the front end developer must carefully consider states and call sequences. To address this issue we simplified the interaction with the service architecture through a Javascript framework (misviewer.js cf. Fig. 1). In the first step, the framework persists the user interactions in an object tree. Then, a request to the service is made based on this object tree. Finally, if a new image has been generated, the framework calls registered clients, effectively implementing the observer pattern (Fig. 2).

Considering that the front end uses native web technologies only, the frame rate achieved is satisfying. To further enhance the user experience, optimizations such as the caching of several requests in a short duration can be applied. Conclusion Web-based DICOM viewers allow radiologists in different locations to collaborate. However, current solutions either lack a common file base or sufficient image processing capabilities. With the presented architecture we showed how a web-based DICOM viewer with features like efficient navigation, organ outlines etc. can be realized. Through this approach, the client side may be implemented using native web technologies only. The architecture presented is used in the GoSmart environment, which is a planning tool for minimally invasive cancer treatments. Acknowledgments This research was funded by the European Commission, under Grant Agreement no. 600641, FP7 Project Go-Smart. The authors gratefully acknowledge the significant contribution to this project made by our clinical partners at Leipzig University, Medical University of Graz, Radboud University Medical Centre Nijmegen and University Hospital Frankfurt. References [1] Haehn D, Rannou N, Ahtam B, Grant E, Pienaar R (2012) Neuroimaging in the Browser using the X Toolkit. 5th INCF Congress of Neuroinformatics. [2] Jodogne S, Bernard C, Devillers M, Lenaerts E, Coucke P (2013) Orthanc—A lightweight, restful DICOM server for healthcare and medical research. IEEE 10th International Symposium: 190–193 [3] Weir P, Reuter D, Ellerweg R, Alhonnoro T, Pollari M, Voglreiter P, Mariappan P, Flanagan R, Park C, Payne S, Staerk E, Voigt P, Moche M, Kolesnik M (2015) Go-Smart: Web-based Computational Modeling of Minimally Invasive Cancer Treatments. CoRR.

Expansion of the regional EHR with XDS and XDS-I on cloud technology to neighbour prefecture and to Homecare H. Kondoh1, T. Kawai2, M. Mochida2, M. Nishimura2, D. Ide3, K. Teramoto1 1 Tottori University Hospital, Medical Informatics, Yonago, Japan 2 SECOM sanin Co.Ltd., Matsue, Japan 3 IBM Japan Co. Ltd., Tokyo, Japan

Fig. 2 The GoSmart environment Results To measure the performance of the service architecture, we ran tests on an average web server (IIS 8; Windows Server 2012; Intel Xeon E5-2665 2.40 GHz; 32 GB RAM). The test data contains three different volumes (small, medium and large in size) and a corresponding mesh for each of these volumes. Table 1 shows the resulting frame rate of the test bed. Table 1 The resulting frame rate of the test bed Dataset

Volume size

Modality Segm. structure

Mesh size

Frame rate

Small volume

16 MB

MR

Liver

864 KB

4.276 fps

Medium volume

31.5 MB CT

Lung

6.705 KB 2.242 fps

Large volume

64 MB

Kidney

417 KB

CT

Performance measurements

4.629 fps

Keywords IHE-XDS  IHE-XDS-I  Virtual server  Cloud server Purpose Our regional EHR expanded to neighbor prefecture and to homecare. The purpose of this paper is to explain and to discuss the way to expand the regional EHR with global standard of XDS and XDS-I and Japanese standard of SS-MIX2 on cloud technology to neighbor prefecture and to Homecare. Methods Our regional EHR sharing system was using global standard of XDS and XDS-I and Japanese standard of SS-MIX2. SS-MIX2 composes text files of HL7 v2.5 and XML files of CDA v2 in each patient’s folder. Japanese EPRs extracted the data in SS-MIX2 as a Japanese standard form. In our system center SS-MIX2 gateway gathered the patient’s SS-MIX2 data and converted to global standard of XDS registry and repository. Our center DICOM gateway gathered the patient’s DICOM data of each hospital and converted to global standard of XDS-I registry and repository. Information provided hospitals increased from eight to fifteen this year. Information reference medical institutions also increased from eight to thirty. To reduce the cost and use the system easily, XDS and XDS-I repository

123

Int J CARS and source servers were set in the center virtual servers. The reference medical institutions expanded to neighbor prefecture and to homecare. Results After the patient was registered in our patient identifier cross reference (PIX) server, SS-MIX Gateway server started to gather the registered patient’s SS-MIX data from the SS-MIX server of each hospital. The DICOM gateway of the center also started to gather the registered patient’s DICOM images from DICOM server of each hospital. The SS-MIX Gateway server converted the SS-MIX data to XDS repository and changed the hospital patient’s ID to the integrated patient’s ID. The DICOM gateway server also changed the hospital patient’s ID of DICOM data to the integrated patient’s ID and transferred them to XDS-I source server. One patient’s data from hospitals were gathered to one XDS repository and one XDS-I source server like as one regional electronic health record (EHR). Each gateways gathered registered patient’s data daily. This year information provided medical institutions were increased. The gateways should be increased twice in virtual server. The licenses of vpn were incresed for increase of users. SS-MIX2 is derived from the HL7, however, it was a problem that laboratory exam code and drug code were not input enough at each hospital. Conclusion We developed the system, which was combined the new system with the old system on SBC cost-effectively and the user operation was similar to the previous system. Because the patient’s data were integrated and stored in one XDS/XDS-I system from different hospitals, they were shown as one EHR. We expected that it seemed to be convenient to get a time series of laboratory results, prescription medicines and image examinations from different hospitals. The long series of laboratory results and prescription medicines seemed to support clinical decisions more efficiently than one hospital series of data. Especially it seemed to be efficient to eliminate redundant prescriptions from several hospitals. Original IHE-XDS and XDS-I system were planned the repository servers are set in hospitals, but we thought it will take more time to show data and the total cost will increase such as distributed storing system. IHE technical frame works permitted to gather repositories. As this system stored the data of EPR and DICOM images in the center server, the storage should be increased in future and it should be a problem. But from the concept of the secure thin client system, data should be gathered. After data were gathered in the center server, data also could be used analytically and efficiently (Fig. 1).

Fig. 1 The dataflows of the system. The lefts are referencing hospitals and the rights are informant hospitals. The upper of the

123

center is portal server, which control connections of thin client servers. The thin client servers shows EPR and PACS viewers of the hospitals and viewer of XDS and XDS-I References [1] Kondoh H, Patient Identifier Cross Reference Server Manages EPR Sharing system. Japanese Journal of Telemedicine and Telecare vol8(2)pp238–241, 2012. [2] Kondoh H, Teramoto K,Kawai T, Mochida M, Nishimur M, Development of the regional EPR and PACS sharing system on the infrastructure of cloud computing technology with server based computing. S61–62, Int. CARS (2012) 7 (Suppl 1):S92–93, 2012. [3] IHE-XDS, IHE-XDS-I: http://www.ihe.net/Technical_Frameworks/ [4] Kondoh H, Expansion of EPR sharing system with SS-MIX2, XDS and XDS-I. Japanese Journal of Telemedicine and Telecare vol9(2)pp132–135, 2013.

Towards an accurate 3D reconstruction of fractured long bones from plain 2D radiographs O. Klima1, P. Kleparnik1, M. Spanel2, P. Zemcik1 1 Brno University of Technology, DCGM, Brno, Czech Republic 2 3Dim Laboratory s.r.o., Brno, Czech Republic Keywords Preoperative planning  Fracture reduction  2D-3D reconstruction  shape prior Purpose Radiographic examinations play an essential role during treatment of traumatized long bones. In case the treatment requires a surgical intervention, a preoperative planning with the aim of the identification of an ideal bone fragments reposition and the best fitting bone plate is commonly involved. Such planning is usually based on 3D models segmented from computed tomography (CT) images of the anatomy of interest. However, the CT examination exposes the patient to higher doses of ionizing radiation and adds more time and costs in comparison to the plain radiographic imaging. Therefore, the possibilities of the preoperative planning based only on plain radiographic images have been investigated in recent years. Reconstruction of the 3D bone shape from the small number of 2D X-ray images is a crucial moment of such planning approach. Most of the reconstruction methods proposed so far focus only on the 2D–3D reconstruction of a single part of the bone and only very few works deal with a 2D–3D reconstruction of the fractured bone [1]. The main goal of this work is a 2D–3D reconstruction with a simultaneous 3D reduction of the fractured bone. The proposed method focuses on the displaced oblique fractures of a femoral shaft. The main contribution of the method is an accurate 3D bone reconstruction and reduction without a prior knowledge of the ground-truth length of the bone. It is assumed that for each bone fragment, X-ray images taken from anterior-posterior and lateral views are available and the relative poses of the radiographs are known. Without loss of generality it is also assumed that each radiograph captures exactly one fully visible fragment of the injured bone. Methods The proposed method consists of two parts. The first part performs the shape reconstruction and works as an intensity-based deformable 2D– 3D registration. It fits a single shape prior of a complete and uninjured femoral bone into the radiographs capturing the individual bone fragments. As a shape prior, the statistical shape and intensity model (SSIM) [2] created from 22 CT images is involved. Beyond the shape variations, the SSIM describes the bone densities using higher-degree Bernstein polynomials, allowing the rendering of digitally

Int J CARS reconstructed radiographs (DRRs) [3]. We formulate the registration as a non-linear least squares problem solved using the iterative Levenberg–Marquardt algorithm [4], which is the well-established optimization method with the high rate of convergence. In the each iteration, the DRRs are rendered from the SSIM, the similarity between the DRRs and the original X-ray images is evaluated using the normalized mutual information (NMI) measure and the poses and the shape parameters of the SSIM are adjusted for the next iteration. The registration is finished when the differences between the original X-ray and DRR images are minimal; the reconstructed 3D model of the patient’s femur is represented by the specific instance of the shape prior. The second part of the proposed method simultaneously performs the 3D bone reduction. As it might be expected, the shape of femoral bones varies mainly in the length, which is independent on other morphometric features of the bone. Therefore, it is not possible to estimate the bone length only by the deformable 2D–3D registration itself. With respect to the assumptions stated above, the key observation is that each vertex of the shape model must belong to exactly one fragment of the bone. Consequently, each vertex must be rendered only in the radiographs depicting the related fragment. As the least squares formulation of the problem allows involvement of multiple metrics, the registration is extended to maximize the count of the SSIM vertices that are assigned to exactly one fragment and rendered in all its DRRs. The maximization ensures the correct estimation of the bone length and the accurate bone reduction. Results The method has been evaluated on a data set created from CT images of 8 people. 12 virtual cases of femoral shaft fractures have been created from each individual, resulting in 96 cases in total. Each case consists of two pairs of the orthogonal virtual X-ray images raycasted from a segmented CT image. For every case, a tested bone was split approximately in the middle of its shaft. A typical test case is illustrated in Fig. 1, the corresponding reconstructed 3D model is depicted in Fig. 2.

Fig. 1 The sample test case. Two pairs of orthogonal radiographs capturing the proximal (left) and the distal (right) part of the virtually fractured femoral bone

Fig. 2 A polygonal model reconstructed from the sample case. The heat map visualizes the differences from the ground-truth model, mean error was 1.53 mm

The initial estimates of the SSIM pose were generated randomly. The bones used for the evaluation were not included in the training set of the SSIM. First, as a baseline solution, only the 2D– 3D registration itself was performed for the test cases. Then the evaluation of the proposed method, including the 3D bone reduction, was performed. The reconstructed 3D models and the groundtruth models segmented from CT images were compared using the symmetric Hausdorff distance [5]. The results are shown in Table 1. Table 1 The average accuracy of the proposed method is sufficient for the purposes of the preoperative planning, while the results of the baseline solution are significantly inaccurate

Proposed method Baseline

Mean distance [mm]

RMS

Maximal distance [mm]

1.38

1.74

7.26

2.52

3.41

11.90

Conclusion We proposed a novel method for the 2D–3D reconstruction of fractured long bones with accuracy sufficient for the application in the preoperative planning. The results clearly confirm that the 2D–3D reconstruction of a fractured long bone must be performed simultaneously with the 3D bone reduction, as the plain deformable registration fails for not being capable of recovering the bone length. With respect to the promising results reached on the synthetic evaluation data set, the ongoing work will focus on the real world cases evaluation. The proposed method is suitable for straight parallelization and consequent acceleration using graphics hardware (GPU), which makes it applicable within the clinical preoperative planning software. Acknowledgement This research has been funded by the Technology Agency of the Czech Republic (TA04011606). References [1] Gong RH, Stewart J, Abolmaesumi P ,,Reduction of multifragment fractures of the distal radius using atlas-based 2d/3d registration‘‘, in Medical Imaging: Visualization, Image-Guided Procedures, and Modeling, Proc. SPIE 7261, SPIE-The International Society for Optical Engineering (2009). [2] Yao J, Taylor RH ,,Construction and simplification of bone density models‘‘, in Medical Imaging: Image Processing, Proc. SPIE 4322, 814–823, SPIE-The International Society for Optical Engineering (2001). [3] Ehlke M, Ramm H, Lamecker H, Hege HC, Zachow S ,,Fast generation of virtual x-ray images for reconstruction of 3d anatomy‘‘, IEEE Transactions on Visualization and Computer Graphics 19, 2673–2682 (2013). [4] Klima O, Kleparnik P, Spanel M, Zemcik P ,,Intensity-based femoral atlas 2D/3D registration using Levenberg–Marquardt optimisation‘‘in Medical Imaging: Biomedical Applications in Molecular, Structural, and Functional Imaging, Proc. SPIE, SPIE-The International Society for Optical Engineering (2016). [5] Aspert N, Santa-cruz D, Ebrahimi T ,,MESH: Measuring Errors between Surfaces using the Hausdorff distance‘‘in Proc. of IEEE International Conference in Multimedia and Expo 2002.

123

Int J CARS

Effects of similar case retrieval for the diagnosis of lung lesions by using computed tomography T. Kotani1, K. Takata2, K. Wakasugi2, K. Kozuka2, K. Kondo2, M. Kiyono2, Y. Nakai1, D. Miwa1, T. Kato1, Y. Ushijima1, T. Sakai3, H. Kimura3 1 Matsushita Memorial Hospital, Radiology, Moriguchi, Osaka, Japan 2 Panasonic Corporation, Advanced research Division, Soraku-gun, Kyoto, Japan 3 University of Fukui, Faculty of Medical Sciences, Yoshidagun, Fukui, Japan Keywords Medical image retrieval  Content-based image retrieval  Clinical evaluation  Similar case Purpose In the recent years, daily clinical operations generate numerous medical images, and therefore, content-based image retrieval systems are required to manage these complex and large imaging databases. We previously proposed a similar case retrieval system for computed tomography (CT) images of diverse lung lesion patterns [1]. When similar cases are retrieved, the search provides incorrect as well as correct diagnosis results, because the textural similarity does not equal a correct diagnosis. Therefore, we need to verify the influence of these search results for the diagnosis of diverse lung lesions on CT, including consolidation and wide-spread ground glass opacity. The purpose of this study was to evaluate the effect of similar cases retrieval for the diagnosis of diverse lung lesion patterns on CT. Methods Subjects. Nine subjects participated in the experiments; all of them were medical doctors, of which 5 were radiology specialists. The experiments were performed from August 16 to August 24, 2015. Apparatus. Two PCs were used: The presentation of the test case and the radiology report were performed by a PC. The other PC presented similar case results to the subjects. The experimenter stood behind the subjects to execute the task presented and operation of the test apparatus. Procedure. The experiment was conducted in two steps as shown in Fig. 1. First, the subjects wrote a radiology report (A) without references. Second, the subjects wrote a report (B) using the similar cases. Three test cases were prepared as examples of having diverse lesion patterns, pneumocystis pneumonia, pulmonary edema, and pulmonary tuberculosis. Age, sex, and all CT slices were presented. In all, 16–19 similar cases were selected for each test, and 30 % of those showed a ‘‘correct’’ diagnosis. The correct diagnosis included appropriate disease identification while reading the images in addition to a definitive diagnosis. These correct diagnoses were determined via a discussion with multiple radiology specialists.

123

Fig. 1 Experimental procedure Results The mean numbers of the correct diagnosis and the accuracy rate of definite diagnosis are shown in Fig. 2. The numbers of correct diagnosis increased to 40 % (1.3 ? 1.8/case) when using similar cases. The accuracy rate of definite diagnosis improved by 8 % with similar case retrieval. These results show that similar case retrieval increased the rate of correct diagnosis; as a result, the precision of definitive diagnosis improved. We then analyzed additional diagnoses using similar cases, and the correlation coefficient between accurate findings and correct diagnosis was 0.6. This result shows the accuracy of diagnosis improves further when similar case retrieval is used if the user obtains the correct findings.

Int J CARS

of the shape. The extracted skeleton points constituted surface group. The branch lines were point group that intersect for the surface by counting connectivity of the voxels. In the second step, 26-connected neighbor thinning was performed to obtain centerlines and center points. To evaluate these extract features, five indexes, the number of the center points, center lines, branch lines, skeleton lines which was comprised of center lines and branch lines, and the maximal length of the skeleton lines were calculated by 26-connected neighbor labeling algorithm (Fig. 1). Fig. 2 The numbers of correct diagnosis and the accuracy rate of definite diagnosis Conclusion We evaluated the effect of similar case retrieval for the diagnosis of diverse lung lesion patterns in the lung on CT. According to the results obtained, we found that the presentation of similar case increased the rate of correct diagnosis; as a result, the accuracy of definitive diagnosis improved. Furthermore, we found the accuracy of diagnosis improved using similar cases, specifically when the user correctly identifies the findings. References [1] Kozuka K, Takata K, Kondo K, Kiyono M, Tanaka M, Sakai T, Kimura H (2013) Development of image-retrieval technology using radiological knowledge extracted from a clinical database of lung CT images, Int J CARS (2013) 8 (Suppl 1):S71–S74.

Assessment of occurrence and growth patterns of cysts in patients with autosomal dominant polycystic kidney disease using volumetric data Y. Matsunaga1, T. Ishii2, T. Igarashi2 1 Chiba University, Graduate School of Engineering, Chiba, Japan 2 Chiba University, Center for Frontier Medical Engineering, Chiba, Japan Keywords ADPKD  3D-thinning  Skeletonization  Morphological analysis Purpose Autosomal dominant polycystic kidney disease (ADPKD) is a congenital disease that deteriorates renal function in accordance with progression of multiple cysts in the renal parenchyma. Though renal volume is an indicator of renal function, it’s difficult to evaluate time to progression into renal failure due to diversity of progression speed of the cysts among patients. Previous studies [1] indicated expanding speed of the renal volume could be determined by two factors, growth and occurrence speed of the cysts, which could be estimated by analyzing developing or distributing patterns of cysts in one slice of CT or MRI images. The more analysis of the development and growth patterns of the mathematical model of cysts in ADPKD needed to advance method of processing volumetric data. However, the three dimensional thinning algorithm rather lacks reliability. The present study aimed to establish a method to the thinning algorithm suitable for comprehensive coverage of whole kidney by calculating three-dimensional shape of the cysts, and verify case of diseases using MRI image by this algorithm. Methods The city block distance transform was computed to the binary data of cysts. Then the two-step three-dimensional thinning algorithms, deleting and retaining voxels depended on comparison of the distance level of mutual voxels, was calculated. The first step used 6-connected neighbor thinning algorithm to extract skeleton points indicating branch lines that became the frame

Fig. 1 Skeleton lines by 3D-thinnning algorithm To evaluate the proposed method, volumetric data consisted of multiple spheres was generated and tested, since each kidney cyst is almost spherical shape. We investigated the changes of the morphological features according to the different number, size, and density. This investigation used simple mathematical models and complicated mathematical models. At last, this algorithm was applied to MRI images that reconstituted volumetric data of several patients with ADPKD (Fig. 2).

Fig. 2 Extraction of skeleton using MRI images of patients with ADPKD

123

Int J CARS

Results In the investigation of the simple mathematical models, the number of the center point, skeleton line were strongly corresponded to the number of the sphere, as well as the maximal lengths of the skeleton was related to the size of sphere. In addition the number of the center point showed the almost same number of spheres. Furthermore, the lengths of the centerlines corresponded to the distance between overlapped spheres. In the investigation of the complex mathematical models, indexes of present study were related to the morphological features of the models too. These indexes could classify the mathematical models of the disease. In the applied to several MRI images with ADPKD, the indexes could be extracted and difference of indexes was revealed between each case of disease. Conclusion The present study proposed an algorithm to extract three-dimensional morphological features of kidney cysts. The proposed method would indicate distribution of number, size and density of the complicated cysts. In the further study, it would be applied to the more MRI data of ADPKD patients to develop novel classification of the progress pattern of the polycystic kidney. References [1] Teranaka S, Ishii T, Matsunaga Y, Sakamoto S, Kamura S, Igarashi T (2015) Quantification of Cysts Development Style for Applying Estimation of Status in Autosomal Dominant Polycystic Kidney Disease, The 103rd Annual Meeting of The Japanese Urological Association, April 2015.

Fully automated segmentation of whole breast in CT images for radiotherapy L. Jiang1,2, P. Li3,2, Q. Li1,2 1 Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai, China 2 Shanghai United Imaging Healthcare Co., Ltd., Shanghai, China 3 University of Technology of Compiegne, Compiegne, France Keywords Whole breast segmentation  Fully automated method  CT images  Breast radiotherapy Purpose Accurate segmentation of whole breast in CT images is an important task in treatment planning system for radiotherapy [1]. Manual or semi-automated segmentation methods are inefficient and may introduce large inter-observer and intra-observer variations [2]. Because the breast parenchyma might be very close to the chest wall line, it should be very challenging to accurately segment dense breast. In this study, we developed and evaluated a fully automated method to accurately segment the whole breast in CT images for radiotherapy. Methods Our fully automated method combined the anatomic and intensity information of breast in CT images, and took advantages of the continuity of the chest wall line across adjacent slices (Fig. 1a–e).

123

Fig. 1 (a) A representative image with the detected skin line (red curve), lung (green curve), the point of cleavage (blue asterisk), the two anterior points of latissimus dorsi muscle (pink circles), the centroid of segmented lung (point ‘‘C’’), and the scheme of the polar coordinate transformation parameters. (b) Polar coordinate transformed image of a band along the chest wall. (c) Binary image of (b) with the threshold of -30. (d) The candidate points for chest wall line with the transition from 0 to 1 in (c). (e) The detected chest all line shown in (b). (f)–(i) Detected boundaries of whole breast with four BI-RADS density ratings (1) Segmentation of skin line and lung—We firstly segmented the skin line and lung using optimal gray-level thresholding method. (2) Identification of anatomic landmarks—We identified the point of cleavage and two anterior points of latissimus dorsi muscle (LDM). The point of cleavage was detected as the lowest point on the uppermiddle portion of the skin line. The distribution of the breast area truncated by the horizontal line passing through the point of cleavage in each slice was used to determine the superior and inferior slices for breast. The two anterior points of LDM were detected as the jumping points from the outer boundary of LDM to the chest wall. These two landmarks were used to determine the lateral borders of the breast. (3) Detection of chest wall line—First, we obtained the polar coordinate transformed image along the chest wall line. Second, threshold of -30 was applied, and the points with transition from 0 to 1 in the binary image were reserved as the candidate points. Third, we traced the chest wall line with the candidate points and interpolated points from the first to the last image column in the polar coordinate transformed image. The detected boundary should be continuous and have similar tendency with the outer boundary of segmented lung. Finally, the detected curve was transformed back to the chest wall line in Cartesian coordinate. The detection of chest wall line started from the central slice and moved gradually to those on both sides. The searching region of the chest wall line was restricted by the detected location of chest wall in the previous slice. Results With the detected skin line, chest wall line, lateral borders, and the superior and inferior slices, we achieved the segmentation of whole breast in CT images. Figure 1f–i illustrated four representative

Int J CARS segmentation results of whole breast with four BI-RADS density ratings (IV—dense: [75 %; III—heterogeneously dense: 50 %– 75 %; II—scattered: 25 %–50 %; I—fatty: \25 %). We obtained 75 clinical chest CT image volumes from 25 patients to evaluate the segmentation method. The three dimensional CT images were with the size of 512 9 512 9 (42–413). The pixel size in x, y axis is 0.684 mm, and that in z axis is 5 mm or 1.5 mm. Compared with the manually delineated regions of breast in 10 cases, our method achieved Dice overlap measure of 89.1 % ± 2.3 % (mean ± SD), and the average deviation distances of 0.14 ± 0.06 mm and 0.58 ± 0.29 mm for the breast skin line and chest wall line, respectively. In addition, the segmentation accuracy was subjectively evaluated by two medical experts to indicate the performance of breast segmentation. The segmentation results of all cases in our dataset were rated ‘‘good’’ so that (almost) no manual revision is needed in clinical practice. Conclusion In this study, a fully automated method for accurate segmentation of whole breast in CT images was developed and evaluated. This accurate segmentation method would be useful for developing fully automated treatment planning system for breast radiotherapy. In the future, we will focus on fully automated segmentation scheme development for fibroglandular tissues and abnormalities inside the breast region in CT images, which are also essential components in breast radiotherapy. Acknowledgements This work was supported by the National Natural Science Foundation of China (Grant Nos. 81301282 and 81471662), and the Science and Technology Committee of Shanghai City (Grant Nos. 13DZ2250300). References [1] Ahunbay EE, Chen GP, Thatcher S, Jursinic PA, White J, Albano K, Li XA (2007) Direct aperture optimization-based intensity-modulated radiotherapy for whole breast irradiation. Int J Radiat Oncol Biol Phys. 67(4):1248–1258. [2] Hurkmans CW, Borger JH, Pieters BR, Russell NS, Jansen EP, Mijnheer BJ (2001) Variability in target volume delineation on CT scans of the breast. Int J Radiat Oncol Biol Phys. 50(5):1366–1372.

Evaluation of human–computer in interventional radiology

interaction

techniques

J. Hettig1, P. Saalfeld1, M. Luz1, M. Skalej2, C. Hansen1 1 Otto-von-Guericke University, Faculty of Computer Science, Germany 2 University Hospital Magdeburg, Clinic of Neuroradiology, Magdeburg, Germany Keywords Human–Computer interaction  Radiology  User study  Freehand gesture Purpose The interaction with intra-operatively acquired radiological image data and volume renderings during radiological interventions is a challenging task. In many cases, medical users have to change their position to control a medical software (using a touchscreen or joystick) or delegate the control to a medical assistant in the operating room or a nearby control room. These approaches can be time-consuming, inefficient, error-prone, and interrupt the workflow. To overcome these limitations, several freehand gesture interaction techniques for sterile environments are proposed. However, the value of these new approaches compared to established interaction techniques such as joystick interaction or delegation to an assistant (mouse and keyboard control) remains unclear.

Methods We conducted a quantitative user study to compare interaction with the Myo gesture control armband (Thalmic Labs Inc.), the Leap Motion Controller (Leap Motion, Inc.), against clinical standards, i.e., task delegation to a medical assistant, and joystick control panel of an interventional angiography system (Artis zeego, Siemens). The study design was derived from a clinical workflow of a diagnostic neuroradiological intervention to identify and characterize pathological structures. Thus, we simulated an intervention in a controlled and repeatable environment setting (see Fig. 1). Two frequently used interaction tasks during a diagnosis were selected for our study.

Fig. 1 Laboratory setup for the user study. On the operating table is an angiographic head phantom to simulate catheter placement. On the large screen radiological image data and volume renderings are displayed First, the interaction with temporal 2D slices (distribution of contrast agent in the vessels) to select an overlay image (selection task). Second, the interaction with a 3D volume rendering (generated from digital subtraction angiography) to adjust the projection plane for the following catheter placement (rotation task). To mimic a real workflow in our laboratory, we used a head phantom wherein the subject had to place a catheter and administer a contrast agent for fluoroscopic imaging. The study was composed of four passes (one for each input device) in which the subject had to perform a (simulated) diagnostic intervention, which includes catheter placement to administer a contrast agent and interaction with the radiological image data. The sequence of the used input device was balanced across participants. After every interaction task, the subject had to answer a single ease question (SEQ) to assess the difficulty of the task. At the end of each pass the subject additionally had to fill in a NASA Task Load Index (TLX) questionnaire to assess the perceived workload. Simultaneously, the time to fulfill each task was measured. Results We present a descriptive data analysis for the task duration, task complexity and subjective task load. A total of six medical students and three radiologists participated. To perform the selection task, participants needed most of the time to use the joystick (m = 39 s, SD = 19 s). They spend less time using the Leap Motion Controller and delegating the task. The Myo armband was the fastest input method (m = 16 s, SD = 2 s). In contrast, the rotation task required the most time using the Leap Motion Controller (m = 141 s, SD = 91 s), followed by the other input devices with roughly the same time. The participants performed this task fastest using the joystick (m = 59 s, SD = 37 s). The subjective assessment regarding the difficulty of the selection task reflects the task performance. Participants stated that selecting an

123

Int J CARS overlay image was most difficult using the joystick. They perceived the interaction with the Myo armband was the easiest way to do it. In contrast, concerning the difficulty of the rotation task the assessment does not reflect the results of task performance and is even contradictory. Accordingly, the rotation task was most difficult to perform using the Myo armband followed by the joystick and the Leap Motion Controller. This task was easiest delegating to an assistant. The highest workload was perceived using the Myo armband. The difference to other interaction methods was especially notable on the dimension physical demand showing that using the Myo armband is physically exhausting. This result is in accordance with the difficulty assessment for the adjustment task. For the task delegation, the participant perceived the lowest workload. Conclusion We performed a quantitative comparison study in a laboratory setup to mimic a neuroradiological diagnostic intervention. The study was performed with medical students and radiologists to evaluate interaction time, task complexity and workload of four different interaction modalities. The results show that the new input devices applied have potential to improve the interaction with radiological image data regarding a decreased task duration and reduced communication effort. However, fatigue due to freehand gesture interaction can be a drawback. Also, the standard devices used during interventions are known, therefore, a longer training period is necessary for the new input devices and new interaction paradigms to be used in such interventions. To make meaningful statements about the value of freehand gesture interaction during radiologic interventions, more participants need to be involved in our ongoing study.

Realistic AAA phantoms for evaluation of centerline-based EVAR planning software P. Hoegen1,2, S. Wo¨rz3, M. Mu¨ller-Eschner1,4, W. Liao3, K. Rohr3, P. Geisbu¨sch5, M. Schmitt5, F. Rengier1,2, H.-U. Kauczor1, H. von Tengg-Kobligk1,6 1 University Hospital Heidelberg, Diagnostic and Interventional Radiology, Heidelberg, Germany 2 German Cancer Research Center (DKFZ), Radiology, Heidelberg, Germany 3 University of Heidelberg, BIOQUANT, IPMB, and DKFZ Heidelberg, Bioinformatics and Functional Genomics, Biomedical Computer Vision Group, Heidelberg, Germany 4 University Hospital Frankfurt, Nuclear Medicine, Frankfurt, Germany 5 University Hospital Heidelberg, Vascular and Endovascular Surgery, Heidelberg, Germany 6 Inselspital, University Hospital Bern, Institute for Diagnostic, Interventional and Pediatric Radiology, Bern, Switzerland

thickness demands and thus to repeated radiation dose exposure. As many AAA patients also suffer from renal impairment, reduced contrast medium administration is desirable. Our aim was to design aortic phantoms mimicking realistic AAAs with exactly defined geometry and centerlines, to evaluate performance of different software and to determine the impact of slice thickness and contrast medium concentration on centerline-based analysis. Methods Three rigid phantoms were constructed using computer aided design (CAD) software. Twelve spherical fiducial markers were integrated for registration of DICOM data with CAD ground truth. Phantoms were built using a 3D printer (resolution: 0.254 mm) and filled with two mixtures of saline solution and iodinated contrast agent (350 mg iodine/ml) to achieve mean density values of approximately 300 Hounsfield units (HU) and 250 HU. Dual source CT imaging was performed using a clinical multislice CT scanner. Slice thickness/increments of 1.0/0.7 mm and 3.0/ 2.0 mm were reconstructed. Image data was analyzed using three different specialized commercial workstations (COM1, COM2 and COM3) as well as in-house developed solutions with a model-based approach (MB [4]) and region growing (RG) and level sets (LS) as standard methods. Automatic centerline analysis was performed without user modification. 3D centerline coordinates and diameters perpendicular to the centerline in 0.5 or 1.0 mm steps were exported. An in-house quantitative evaluation framework was used for evaluation [4]. CAD ground truth, DICOM data and analysis results were aligned using a landmarkbased registration scheme. Centerline positions, lengths and minimal and maximal diameters were evaluated w.r.t. ground truth. Statistical analysis was performed using established statistics software for two-sided t-tests. Results Significant deviation from centerline ground truth was observed with all software (p \ 0.001 each). Differences between software were significant (p \ 0.001, respectively) except for COM2 vs. MB (p = 0.31). Centerline position, minimal and maximal diameter and total centerline length errors for different software are shown in Table 1. Maximal diameters could not be measured with COM1. RG and LS were not considered for length evaluation due to different post-processing impairing direct comparability. Evaluation results for different slice thicknesses and contrast medium concentrations are shown in Table 2. Table 1

Errors for different software (all results given as mean value ± standard deviation) COM1

123

COM3

MB

RG

LS

Centerline position [mm]

3.2 ± 2.9

0.7 ± 0.6

1.8 ± 1.6

0.7 ± 0.8

1.0 ± 1.6

0.9 ± 1.3

Minimal diameter [mm]

1.5 ± 1.2

0.8 ± 1.0

8.7 ± 11.2

0.7 ± 1.3

2.8 ± 1.8

1.6 ± 1.3

Maximal diameter [mm]

n/a

0.7 ± 2.0

9.6 ± 12.6

0.8 ± 1.8

1.5 ± 2.2

0.8 ± 2.0

Total length [mm]

-17.3 ± 4.2

-5.6 ± 1.6

-12.7 ± 4.0

5.5 ± 6.3

n/a

n/a

Table 2

Errors depending on imaging parameters (all results given as mean value ± standard deviation)

Keywords EVAR  Phantom  Software evaluation  Aortic aneurysm Purpose Centerline-based assessment of computed tomography (CT) angiography is state-of-the-art for preinterventional planning of endovascular aortic repair (EVAR) in patients with abdominal aortic aneurysm (AAA) [1]. Many clinically established and research-purpose centerline analysis software solutions exist [1]. These have been validated with clinical data lacking known ground truth [1, 2], highly simplified phantoms not mimicking realistic pathologies [3] or virtual models. The lack of phantoms providing both realistic and exactly defined ground truth was also stated in recent studies [2]. In addition, different approaches have been compared rarely and only in patient data without known ground truth. Although a slice thickness of 1 mm or less is recommended for aortic centerline analysis, imaging outside of vascular centers is often performed using thicker slices, leading to repeated imaging to match slice

COM2

Slice thickness/increment 1.0/0.7 mm

3.0/2.0 mm

Contrast medium p-value

300 HU

250 HU

p-value

Centerline position [mm]

1.3 ± 1.6

1.5 ± 2.2

\0.001

1.4 ± 1.9

1.4 ± 1.9

0.22

Minimal diameter [mm]

2.6 ± 6.0

2.9 ± 5.3

\0.001

2.7 ± 5.6

2.9 ± 5.7

\0.001

Maximal diameter [mm]

2.6 ± 6.7

2.7 ± 6.9

0.23

2.5 ± 6.3

2.8 ± 7.2

0.001

Total length [mm]

8.0 ± 22.6

6.9 ± 25.1

0.82

6.5 ± 22.2

8.4 ± 25.5

0.7

Conclusion Evaluated software showed significant differences in automatic centerline-based analysis. Slice thickness influences centerline coordinates and diameter measurements. Contrast medium concentration does not significantly influence centerline localization, but diameter measurements. Length measurements are robust with regard

Int J CARS to slice thickness and contrast medium concentration. Choice of planning software may influence measurement results, EVAR planning, stent graft configuration and potentially even clinical outcome. References [1] Corriere MA, Islam A, Craven TE, Conlee TD, Hurie JB, Edwards MS (2014) Influence of computed tomography angiography reconstruction software on anatomic measurements and endograft component selection for endovascular abdominal aortic aneurysm repair. J Vasc Surg 59(5):1224–31. [2] Ghatwary T, Karthikesalingam A, Patterson B, Hinchliffe R, Morgan R, Loftus I, Salem A, Thompson MM, Holt PJ (2012) St George’s Vascular Institute Protocol: an accurate and reproducible methodology to enable comprehensive characterization of infrarenal abdominal aortic aneurysm morphology in clinical and research applications. J Endovasc Ther 19(3):400–14. [3] Martinez-Mera JA, Tahoces PG, Carreira JM, Suarez-Cuenca JJ, Souto M (2015) Automatic characterization of thoracic aortic aneurysms from CT images. Comput Biol Med 57:74–83. [4] Wo¨rz S, Hoegen P, Liao W, Mu¨ller-Eschner M, Kauczor HU, von Tengg-Kobligk H, Rohr K (2016) Framework for Quantitative Evaluation of 3D Vessel Segmentation Approaches using Vascular Phantoms in Conjunction with 3D Landmark Localization and Registration. Proc. SPIE Medical Imaging 2016: Image Processing (MI’16), accepted for publication.

RFA guardian: comprehensive simulation of the clinical workflow for patient specific planning, guidance and validation of RFA treatment of liver tumors P. Voglreiter1, P. Mariappan2, T. Alhonnoro3, H. Busse4, P. Weir2, M.Pollari3, R.Flanagan2, M. Hofmann1, D. Seider4, P. Brandmaier4, M.J. van Amerongen5, R. Rautio7, S. Jenniskens5, R. Blanco Sequeiros7, R.H. Portugaller6, P. Stiegler6, J. Fu¨tterer5, D. Schmalstieg1, M. Kolesnik8, M. Moche4 1 Graz University of Technology, Institute for Computer Graphics and Vision, Graz, Austria. 2 NUMA Engineering Services Ltd., Dundalk, Ireland. 3 Aalto University, Dep. of Neuroscience and Biomedical Engineering, Helsinki, Finland 4 Leipzig University Hospital, Dep. of Diagnostic and Interventional Radiology, Leipzig, Germany 5 Radbound University Nijmegen Medical Center, Nijmegen, Netherlands 6 University Clinic of Radiology Graz, Austria 7 Turku University Hospital, Medical Imaging Center of Southwest Finland, Turku, Finland 8 Fraunhofer Institute for Applied Information Technology, Sankt Augustin, Germany Keywords Radiofrequency ablation  GPGPU  Clinical workflow modelling  Patient-specific treatment planning Purpose Radio frequency ablation (RFA) of liver malignancies emerged as alternative to open surgical resection. Planning RFA of liver tumors is a complex task, sometimes even for experienced interventional radiologists (IRs), because many patient-specific or site-characteristic parameters have not been considered in previously defined workflows [1, 2]. Therefore, a multi-center analysis of the medical workflow and translating it into an application capable of patient-specific planning is necessary for optimizing the treatment. Further, easy integration of the application into the medical workflow is a key element. Therefore, minimized demand in terms of infrastructure, but also usability and computational performance of the application, are critical aspects.

Methods The proposed RFA Guardian incorporates three phases: (1) pre-interventional processing to create patient-specific models, based on images recorded during standard treatment; (2) interventional parameterization for probe positioning and massively parallel RFA simulation on commodity GPUs and (3) post-interventional validation for determining the treatment results. A variety of parameters allows for utterly flexible anatomical modelling of the patient’s liver, tumor, vessels but also exact needle parameterization. Furthermore, a sophisticated bio-heat model [3], encompassing patient-specific perfusion measurement of tumors and healthy tissue, specific heat capacity and thermal conductivity, contributes to accurate prediction of the size and shape of the RFA lesion. The RFA Guardian exploits the computing power of readily available consumer hardware, including parallelized CPU-based methods for segmentation and registration of patient images, and massively parallel computing on the GPU for fast simulation. While pre-interventional modelling requires some careful interaction for refining and optimizing the input for the simulation procedure, interventional parameterization is generally fast. This allows for quick exploration of the parameter space with respect to the invariant preinterventional model, and even in situ evaluation of, e.g., needle placement, for optimizing tumor coverage. Needle identification on interventional scans and registration into the pre-interventional model only require determination of a few points, acting as landmarks, in the images. In case of relocating the needle, a considerable portion of these points can remain invariant and, in combination with the fast simulation approach, predicting the outcome of a given parameterization takes less than 10 min in a worst case scenario. Results The generalized clinical workflow is translated into a user-centered application for predicting, planning and validating RFA treatment. Evaluation in a retrospective study on 18 real cases (Table 1) showed promising results in terms of computational performance and accuracy. The almost fully automatic pre-interventional modelling, including automatic liver segmentation, automatic registration of contrast-enhanced CT images, automatic vessel segmentation, semi-automatic tumor segmentation and automatic generation of a Finite Element Mesh takes up to an hour of mostly computation time in complex cases. In contrast, the interventional phase was designed to exploit feedback of the user and perform much faster. With pure computation times of 2–6 min for simulating real heating protocols of up to 60 min, the employed algorithm performs up to an order of magnitude faster than real-time heating in a high-resolution simulation domain. In terms of accuracy, the deviation between real and simulated lesion ranges between 1–3 mm for cases with little to no track ablation, Track ablations are not part of the computational model due to their inconsistencies and ambiguities. Table 1 Volume and Surface Deviation comparing predicted lesion and real lesion. Results were obtained during a retrospective study, encompassing 18 cases from the clinical routine Dice score Mean StdDev

Rel. vol. deviation (%)

Surface deviation (mm)

70.09

13.61

2.42

8.84

12.24

0.80

Minimum

44.29

0.03

0.96

Maximum

86.64

36.75

4.57

Feedback from four medical sites carrying out the retrospective evaluation suggests a high degree of usability and performance, leading to increased acceptance in the medical practice.

123

Int J CARS Conclusion An innovative, integrated software environment for patient-specific planning, validation and guidance of the RFA treatment of liver tumors, based on registration, segmentation and simulation algorithms, is presented. A first multi-center retrospective clinical trial proves the functionality and accuracy of the tool within the clinical workflow. Acknowledgements This work was co-funded by the European Union’s Seventh Framework Programme under the grant numbers 600641 and 610886. References [1] Kerbl B, Voglreiter P, Khlebnikov R, Schmalstieg D, Seider D, Moche M, Stiegler P, Portugaller R, Kainz B (2012) ‘‘Intervention planning of hepatocellular carcinoma radiofrequency ablations’’. Clinical Image-Based Procedures. From Planning to Intervention:9–16. [2] Schumann C, Rieder C, Bieberstein J, Weihusen A, Sidowitz S, Moltz JH, Preusser T (2010) ‘‘State of the art in computer assisted planning, intervention and assesment of liver-tumor ablation’’, Crit. Reviews in Biomedical Engineering, pp. 38(1):31–52. [3] Pennes, HH (1998) ‘‘Analysis of tissue and arterial blood temperature in the resting human forearm,’’ J. Appl. Physiol, pp. 85(1):5–34.

robot is designed to imitate real motion of inserting and rotating the guidewire by two fingers. The operator grips the handle and manipulates it by two fingers. The dynamic model of the haptic device is derived. The inertia compensation control (ICC) for the dynamic inertial load and friction is implemented. Two pressure sensors are attached at both walls of the haptic device to detect the motion direction of the operator as shown in Fig. 1(a). Also, the force feedback is implemented by using a force signal measured at the dc motor controller of the slave robot. The system validation is conducd by experiments using a cardiovascular phantom immersed in a water tank. The performance of the system is determined by two criteria; how well the guidewire is steered in the artery and how precisely the guidewire approaches the target place. Figure 1(b) shows the configuration of the master/slave system for the experiment.

Fig. 1 The master/slave system Design and implementation of haptic robotic system for vascular intervention K. Y. Jung1, H.-J. Cha1, B.-J. Yi1 1 Hanyang univ., Ansan-si, South Korea Keywords Master-slave robotic system  Vascular intervention  Catheter  Chronic total occlusion Purpose Vascular intervention (VI) is a therapy for vascular disease using a guidewire and a catheter. During VI, an operator can check the target arteries and the position of the VI devices under the X-ray. The human body has plenty of curved vascular branches, so an operator manipulates a guidewire and catheter delicately by feeling of their fingertips. This procedure usually takes long time. Therefore, both surgeons and patients are irradiated too much. For this reason, it is important to decrease treatment time and reduce radiation exposure by some means. To cope with such problem, a robotic approach which can reduce radiation exposure through a master-slave system and help the devices to be controlled precisely is necessary [1, 2]. It is also important to offer the haptic feedback to the operator because the operator depends on feeling of fingertips during VI. The purpose of this study is to propose a robotic system which offers the haptic feedback and assists VI in aspects of less X-ray exposure and convenient user interface. Methods We already have developed a master/slave system for VI [3]. The system has 4 DOF; an insertion and a rotation motion for a catheter and a guidewire. The VI devices are inserted via femoral artery which is straight and thick, so insertion of devices from the femoral artery to the arterial branch is easy. After that, the guidewire is manipulated delicately near the target arterial branch to make a path for the catheter, and the catheter is easily inserted along the guidewire. In this paper, therefore, a master/slave system having only 2 DOF for the guidewire is proposed for compact design and accurate control of the guidewire. Both a master robot and a slave robot have 2 DOF for rotation and insertion of the guide wire. The slave robot is designed to implement the motion of the guidewire by using a roller mechanism. The master

123

Results Figure 2(a) and (b) are the case that the ICC is not applied and applied, respectively. The top figures denote the position information of the master device, and the middle figures denote the value of pressure sensors attached to both walls of the handle. The bottom figures are current measured at motor of the haptic device. It is noted that in Fig. 2(b), the handle wall is not under pressure except for the moment when the direction changes, while in Fig. 2(a), the handle wall is under pressure continuously during the motion. According to this result, it was verified that the operator could feel lighter when applying the ICC, because the measured pressure is reduced.

Fig. 2 Experimental results Figure 2(c) is the measured load at the dc motor of the slave robot when the guidewire is inserted into the phantom. The top and bottom figures, respectively, denote the case of the unblocked artery and the blocked artery. A solid line indicates the position of the master device, and a dotted line indicates the load measured at the motor of the slave device. In case of the blocked case, the current was increased up to about 600 mA. On the other hand, in case of the unblocked case, the current was ranged from 200 mA to 300 mA. The measured load signal in the slave device was used as a reflection force

Int J CARS at the haptic device. And operator can adjust the sensitivity of the haptic feeling by changing the proper gain. Figure 2(d) shows the experimental results with the treatment process. Operator positions the catheter and the guidewire to the initial position of the cardiovascular phantom manually. Operator steers the guidewire using the master/slave system. According to the results of the experiment, the proposed system has achieved a successful navigation inside the artery like the conventional process. Conclusion The haptic robotic system for vascular intervention was proposed. The ICC was conducted to reduce the effects of inertial load. Then the effectiveness of such control method was verified through experiment. And then, another experiment is performed for haptic feedback control. It was confirmed that the measured current at the slave controller can be used for haptic feedback signal at the master side. Steering performance test was also performed through interlocking between master and slave device. References [1] http://www.hansenmedical.com [2] Fu Y, Gao A, Liu H, Li K, Liang Z (2011) Development of a novel robotic catheter system for endovascular minimally invasive surgery, Proc. IEEE/ICME Int. Conf. on Complex Med. Eng. pp. 400–405. [3] Cha HJ, Jung KY, Yi BJ, Won JY (2015) Assembly Type Robotic System Design for Vascular and Interventional Radiology (VIR), Asian Conf. on Computer Aided Surgery.

The ABCs of precision nuclear medicine therapy for prostate cancer: alphas, betas and the camera C. Wright1, C. Odom2, N. Hall3, J. P. Monk4, A. Mortazavi4, M. Knopp1 1 The Ohio State University, Wright Center of Innovation, Department of Radiology, Columbus, United States 2 The Ohio State University Wexner Medical Center, Department of Radiology, Columbus, United States 3 Veterans Affairs Medical Center, Philadelphia, United States 4 The Ohio State University Wexner Medical Center, Internal Medicine, Columbus, United States Keywords Prostate cancer  Radiotherapy  Portable gamma camera  Nuclear medicine Purpose Precision nuclear medicine is the concept of adapting clinical practices and specific disease treatments to individual patients. In particular, new applications for existing nuclear medicine technology that can provide real-time, immediate feedback to physicians during patient treatment is the primary goal. Targeted radiotherapy is one clinical approach for treating malignant/metastatic lesions using intravascular administration of therapeutic radioisotopes, peptides, antibodies and microspheres. For example, bone metastases in prostate cancer patients are readily amenable to targeted radiotherapy using intravenous Radium-223 (223Ra) and Samarium-153 (153Sm). 223Ra is an alpha-particle emitting radioisotope that mimics calcium and forms complexes with hydroxyapatite in areas of increased bone turnover, such as bone metastases. Likewise, 153Sm is a beta-particle emitting radioisotope which also binds to hydroxyapatite in bone metastases. Both have the advantage of maximizing local radiation effects to bony metastatic lesions while minimizing radiation toxicity to adjacent normal bone and soft tissues. Additional radiation is also generated by 223Ra and 153Sm which produces photons that can be readily detected and imaged using conventional portable gamma camera (PGC) systems. The purpose of this study is to evaluate to the capabilities of existing portable gamma camera technology to provide real-time feedback to treating physicians before, during

and after targeted radiotherapy administrations that can enable new precision nuclear medicine practices. Methods A retrospective review was performed in prostate cancer patients who received targeted radiotherapy (i.e., 223Ra or 153Sm) for bone metastases and who were also simultaneously imaged using a large field-ofview portable gamma camera (Ergo, DigiRad) for purposes of quality assurance and quality control. Prior to radiotherapy administration, imaging of the injection site and patient’s chest was performed to establish background radioactivity counts. During radiotherapy administration, dynamic imaging of the chest was performed so that the treating physician could verify in real-time systemic circulation of the therapeutic radioisotope during the entire intravenous administration. After radiotherapy administration and removal of the patient’s intravenous access, additional imaging of the patient’s intravenous access site in the arm was performed to assess for any focal radioisotope extravasation. The injection syringe containing the therapeutic radioisotope was also imaged before and after administration to assess residual radioactivity. Quantitative region of interest analysis was performed in each case to assess radioactivity counts and verify that time from the start of injection to peak systemic chest activity for 223Ra and 153Sm administrations were at least 1 min. Results A total of 30 targeted radiotherapy administrations using 223Ra and 153 Sm were included. Real-time dynamic imaging of the anterior chest allowed the treating physicians to visually confirm systemic circulation of the therapeutic radioisotope early during the intravenous injection period. Subsequent quantitative region of interest analysis of the dynamic chest imaging also confirmed at least 1 min injection to peak systemic radioactivity times (i.e., appropriate intravenous injection rates by the treating physicians). No instances of focal therapeutic radioisotope extravasation into the soft tissues of the injection arm or delivery device failure were noted. Conclusion These results demonstrate that current portable gamma camera imaging systems can be easily adapted and clinically used for the realtime monitoring and assessment of various radiotherapy administrations (Alpha, Beta, Gamma). This innovative PGC application provides immediate visual feedback to the treating physician that the intravenous radiotherapy administration is proceeding normally. On the other hand, non-visualization of early systemic radioactivity during the injection may suggest that a technical malfunction has occurred in the delivery system or there is a focal extravasation at the intravenous access site. In both instances, the treating physician would stop the injection and quickly use the PGC to visually survey the patient’s intravenous access site for extravasation or the delivery system for evidence of a leak. New applications for existing nuclear medicine technologies to advance precision nuclear medicine practices represent a tremendous clinical opportunity for physicians and patients.

Metric learning for TNM classifications of patients with head and neck tumors K. Birnbaum1, V. Zebralla2, A. Boehm2, A. Dietz2, T. Neumuth1 1 Leipzig University, Innovation Center Computer Assisted Surgery, Leipzig, Germany 2 University Medical Center Leipzig, Department of ENT Surgery, Leipzig, Germany Keywords Metric learning  TNM classifications  ENT surgery  Large margin nearest neighbor Purpose To achieve the best possible outcomes for patients with head and neck tumors, the optimal selection of appropriate treatment is essential.

123

Int J CARS

The description of the spread of cancer is the basis for all further treatment decisions. Gold standard for the systematic classification of malignant tumors are TNM classifications. They can be grouped according to the specifications of the Union for International Cancer Control (UICC) to stages, wherein each stage reflects the survival rate for the respective type of cancer. To illustrate this, Table 1 shows the assignment of TNM classifications to the corresponding UICC stages for oro- and hypopharyngeal tumors [1]. Conventional metrics for strings as the edit distance by Levenshtein [2] would assign similar or even equal distances between TNM classifications of completely different UICC stages (e.g. one for T1N0M0 and T2N0M0 as well as T1N0M0 and T1N0M1). Hence, for a useful comparison of TNM classifications a metric is needed, which considers not only the similarity of character strings, but also achieves a good selectivity between tumor classifications of different UICC stages.

Table 1 UICC stage grouping for oro- and hypopharyngeal tumors UICC stage

T

N

M

Stage 0

Tis

N0

M0

Stage I

T1

N0

M0

Stage II

T2

N0

M0

Stage III

T1, T2

N1

M0

T3

N0, N1

M0

Stage IVA Stage IVB Stage IVC

T1, T2, T3

N2

M0

T4a

N0, N1, N2

M0

T4b

every N

M0

every T

N3

M0

every T

every N

M1

Methods In the field of machine learning, supervised metric learning refers to the computation of optimal distance metrics based on labeled data. Recently, these methods have gained considerable interest in the community, because metric learning can significantly improve the performance of standard metric based machine learning algorithms. However, we used a kernel based approach for the representation of the TNM classifications as vectors, similar to the spectrum kernel [3]. In contrast to this method we defined a feature map which is indexed by all possible T, N and M elements for a given problem (e.g. oroand hypopharyngeal cancer), instead of using all contiguous subsequences of a predefined length. The kernel matrix values can then be easily computed by the inner product of the feature vectors. Furthermore, the learning of Mahanalobis distance was realized by the Large Margin Nearest Neighbor (LMNN) method [4] directly on the feature vectors. Training data for the algorithm could be obtained from the finite set of the TNM-combinations as well as the corresponding UICC stages. The visualization of the data was carried out by a 2-dimensional projection using Kernel Principal Component Analysis (KPCA) [5]. Results The method presented here allows for a UICC-specific, quantitative and visual analysis of differences between TNM classifications. Figure 1 shows the two-dimensional projection of the TNM classifications for oro- and hypopharyngeal tumors by KPCA. The left diagram shows the 2D projection without metric learning. The right diagram shows the result after using the LMNN method. It can be clearly seen that the learned metric reaches a better differentiation between the TNM classifications of different UICC stages.

123

Fig. 1 Two-dimensional projection of the TNM classifications by KPCA. Left: 2D projection without metric learning. Right: results after using the LMNN method Conclusion The incorporation of the UICC stage grouping into a metric for strings allows an effective comparison of TNM classifications. Furthermore, the method is applicable to other oncology areas. References [1] C. Wittekind and H.-J. Meyer, TNM: Klassifikation Maligner Tumoren. John Wiley & Sons, 2013. [2] V. I. Levenshtein, ‘‘Binary Codes Capable of Correcting Deletions, Insertions and Reversals,’’ Soviet Physics Doklady, vol. 10, p. 707, Feb. 1966. [3] C. Leslie, E. Eskin, and W. S. Noble, ‘‘The spectrum kernel: a string kernel for SVM protein classification,’’ Pac Symp Biocomput, pp. 564–575, 2002. [4] K. Q. Weinberger and L. K. Saul, ‘‘Distance Metric Learning for Large Margin Nearest Neighbor Classification,’’ J. Mach. Learn. Res., vol. 10, pp. 207–244, Jun. 2009. [5] B. Scholkopf, A. Smola, and K.-R. Mu¨ller, ‘‘Kernel principal component analysis,’’ in Advances in Kernel Methods—Support Vector Learning, 1999, pp. 327–352.

Method for MRI measurement of 3D hip kinematics R. Eveleigh1, Y. Konishi2, K. Masamune2, R. Ellis1 1 Queen’s University, Kingston, Canada 2 Tokyo Women’s Medical University, Tokyo, Japan Keywords MRI  Hip joint  Registration  Kinematics Purpose Recent investigation of hip kinematics in cadavers suggests translation magnitudes of 3 mm for intact cadaver hips and 6 mm for disarticulated cadaver hips [1]. Methods to date have been highly invasive; CT scans were needed for 3D surface models of bony anatomy, and local coordinate reference markers fixed to bone screws were needed to track the bony kinematics. These ex vivo findings are important, but need to be extended to live human subjects. A less invasive method for precisely measuring hip kinematics is needed. Previously, knee kinematics were estimated using high resolution MRI to obtain detailed 3D bone surface anatomy and low resolution MRI to obtain bone kinematics [2]. This method used conventional closed-gantry MRI to obtain the images. Registration of low resolution surface contours to high resolution 3D surface models yielded 3D bone kinematics. This previous method cannot be applied directly to the hip. Hip range of motion (ROM) is seriously constrained in conventional MRI because the lower leg cannot move through the large volume required by hip motion. Open-gantry MRI has potential use for measuring hip kinematics, with a parallel-disc geometry accommodating a much larger hip ROM.

Int J CARS Methods One coronal plane MRI scan (T2-weighted, voxel size 0.7 9 0.7 9 0.8 mm3, Siemens MAGNETOM Skyra, 3T) of the right hip of a healthy adult male was obtained. Cortical and sub-cortical bone surfaces were segmented, from which 3D triangulated meshes were derived to describe the pelvis and proximal femur anatomy. Coronal plane scans (T1-weighted, voxel size 4.0 9 1.3 9 0.8 mm3, Hitachi AIRIS II, 0.3T) of the subject’s hip were acquired in a neutral pose (pose 1). Scans were also acquired in two functional poses: pose 2 was with legs crossed and pose 3 was with toes touching. An open-gantry MRI scanner with a large body coil was used to accommodate a large hip ROM. Scan time was under 6 min. Contours that described pose 1, pose 2, and pose 3 of the femur relative to the pelvis were extracted from segmentations of sub-cortical bone. 3D surface models and corresponding bone contours were imported into MATLAB. After manual pre-alignment of sub-cortical surface models and contours an iterative closest points (ICP) algorithm [3] was used to obtain registrations between corresponding surface models and contour sets for pose 1, pose 2, and pose 3. The resulting transformations were used to compute pose 1, pose 2, and pose 3 of the femur and pelvis cortical surface models derived from the low-resolution images. The processes of surface selection and pose-specific registration are illustrated in Fig. 1. A representative open-gantry MRI image is given in Fig. 2.

Fig. 2 Open-gantry, low resolution MRI of the hip joint. The boundary between pelvis and femur cortical bone can be difficult to distinguish, while sub-cortical bone contours are easily segmented Results Every registration was verified by inspection. Surface model to contour registration root mean squared error (RMSE) for pose 1, neutral, was 1.29 mm for the pelvis and 1.23 mm for the femur. RMSE for pose 2, legs crossed, was 1.49 mm for the pelvis and 1.37 mm for the femur. RMSE for pose 3, toes touching, was 1.47 mm for the pelvis and 1.39 mm for the femur. All RMSE values were less than 1.5 mm. Results are tabulated in Table 1. Table 1 RMSE values for sub-cortical surface model to contour registration Anatomy

Hip joint position Pose 1

Pose 2

Pose 3

Pelvis

1.29

1.49

1.47

Femur

1.23

1.37

1.39

All values are in mm

Fig. 1 (a) Sub-cortical surface models and (b) posed, sub-cortical surface contours are (c) registered to each other, producing a transform to move the cortical surface models to the (d) posed relative orientation

These RMSE values are less than 1/2 of the mean magnitude of hip translations previously measured in intact cadaver hips, and less than 1/4 of the mean translation magnitude measured in cadaver hips disarticulated to the level characteristic of hip replacement surgery. This technique for measuring hip kinematics has potential for use either in future hip replacement surgical planning, or in biomechanical studies of hip joint kinematics in healthy subjects.

123

Int J CARS Conclusion 3D surface models of hip anatomy acquired in a conventional MRI scanner were registered to sub-cortical surface contours acquired in a low-tesla open-gantry MRI scanner. Relative poses of the 3D cortical surface models were computed from sub-cortical surface contours. All registration RMSE values were less than 1/2 of the mean hip translation magnitude found in cadavers. This method shows promise for future studies of millimeter-scale hip motion in live human subjects. Further work to improve this method could include techniques for reducing image acquisition time and/or registration error. These findings are preliminary so a validation study would be useful. Such a study might compare kinematic measurements from this method to an accepted ‘‘gold standard’’ measurement method. This is a first report on the use of multimodality MRI scans to estimate in vivo hip kinematics. References [1] Zakani S, Ellis RE (2012) Tracking translations in the human hip. Proc ASME Int Mech Eng Cong Expo, paper #IMECE2012-87882. [2] Fellows RA, Hill NA, Gill HS, MacIntyre NJ, Harrison MM, Ellis RE, and Wilson DR (2005) Magnetic resonance imaging for in vivo assessment of three-dimensional patellar tracking. J Biomech 38(8):1643–1652. [3] Besl P, McKay N (1992) A Method for Registration of 3-D Shapes IEEE Trans Pattern Anal Machine Intell 14(2):239–256.

(ROI) of the dura mater (Fig. 1). Muscles were detached from their cranial origin and attached to an electronic scale. The skull was fixed to a platform allowing static placement of the cervical spine with respect to a stable skull. Static loads were applied to individual muscle up to 20.0 N in increments of 2.5 N.

Structure light scanning technology for soft tissue displacement

The 3D geometry of the ROI was acquired using a handheld structured-light scanner (Artec Spider, Artec Group, Luxembourg) with reported 3D resolution of 0.1 mm, 3D point accuracy of 0.05 mm and point acquisition speed of up to 1,000,000 points/sec. Scanning coverage was increased by aligning the 3D points in realtime using Artec Studio 9.2 software. Scans were acquired at a working distance of 17–30 cm and contained 170 ± 70 frames, obtained at 7 frames per second. Each loading state scans was aligned and fused into a single triangulated mesh. After software de-noising and smoothing, the triangulated mesh was exported as a stereolithography file. Models were imported and aligned by a paired-point registration on external landmarks present in the scans. The unloaded hyperextended state was compared to the unloaded neutral state, and each loading state in hyperextension was compared to the unloaded hyperextended state (Fig. 2). Distance residuals between aligned meshes were measured using custom software. The 95th percentile of displacement was reported, to exclude outliers.

1

2

3

2,3,1

G. Venne , B. J. Rasquinha , M. Kunz , R. E. Ellis 1 Queen’s University, Department of Biomedical and Molecular Sciences, Kingston, Canada 2 Queen’s University, Department of Mechanical and Materials Engineering, Kingston, Canada 3 Queen’s University, School of Computing, Kingston, Canada Keywords Image processing  Biomechanics  Musculoskeletal  Structured light Purpose Measuring soft tissue deformation and displacement, during surgery or biomechanical testing, is technically challenging. Medical imaging for evaluation of soft tissue behaviour under different mechanical constraints, such as MRI, CT or ultrasound [1, 2], has practical limitations in many experimental settings. Laser range scanners have been used to acquire registration data and to assess large-scale tissue deformation [3], but may be limited by cost and complexity [4]. Handheld structured-light scanning is proposed as an alternative sensing modality. This relatively new technology offers improvements in cost, ease of use and accuracy; some commercial scanners reporting a point accuracy of 0.05 mm. A real-time 3D cloud of points of the scanned surface is created by projecting a known pattern of light onto the object and measuring the reflection. Accuracy and precision of structured-light scanning have been validated for facial soft-tissue morphology recording in clinical and research applications [5]. We hypothesize that structured-light scanning can accurately capture soft tissue displacement in a biomechanical experiment. We used such a scanner to measure spinal dura mater displacement under a series of loads applied to a suboccipital muscle, the rectus capitis posterior minor (RCPMi). This muscle is known for having a connective tissue link to the spinal dura mater at the atlanto-occipital level, but it is not clear whether this link has a mechanical role. Methods With IRB approval, 5 fresh-frozen cadaveric heads-necks were acquired. The RCPMi were exposed as was the Region of Interest

123

Fig. 1 a) Superior view of skull and ROI. b) Superior view of mounted skull onto the testing platform; nuts were used as reference structures for accuracy validation measures. c) Photograph of the ROI at hyperextended position. d) Scan of the ROI at hyperextended position. e) Scan at neutral position

Fig. 2 Topographic map of compared scans showing displacement magnitude

Int J CARS The scanner accuracy was verified by comparing 12 virtual measurements of reference structures in the scanning field to physical measurement taken with calipers. Results The validation tests yielded an average deviation of 0.004 ± 0.14 mm, with a maximal deviation of 0.26 mm between caliper and scanner measurements; this agreed with the manufacturer’s specifications. Anatomically, the dura mater bulged anteriorly, inside the vertebral canal by 3.1 ± 0.8 mm during hyperextension when compared to neutral spine position (Fig. 1). Increased loadings on the RCPMi in the hyperextended pose amplified the posterior displacement of the ROI in a quasi-linear relationship. The dura mater displaced 1.6 ± 0.5 mm at 20.0 N. The displacement during muscle loading reduced the dura mater bulging caused by hyperextension positioning by 53.4 % ± 6.9 %. Conclusion Structured-light scanning technology accurately evaluated soft-tissue deformation and displacement in a biomechanical experiment. Displacement of the dura mater at the atlanto-occipital level was approximately 4 mm, which suggests that the RCPMi muscles mechanically influence the dura mater at the atlano-occipital level by reducing the bulging into the spinal canal during head and neck hyperextension. Handheld structured-light scanning technology could potentially be used in other soft-tissue surface deformation and displacement evaluation, in biomechanical experiments and in intraoperative use. References [1] Cevidanes LHC, Motta A, Proffit WR, Ackerman JL, Styner M (2010) Cranial base superimposition for 3-dimensional evaluation of soft-tissue changes. Am J Orthod Dentofac Orthop 137:S120-9. doi:10.1016/j.ajodo.2009.04.021. [2] Ahmadian A, Serej ND, Karimifard S, Farnia P (2013) An Efficient Method for Estimating Soft Tissue Deformation Based on Intraoperative Stereo Image Features and Point-Based Registration. Int J Imaging Syst Technol 23:294–303. doi: 10.1002/ima.22064. doi:10.1002/ima.22064. [3] Rucker D, Wu Y, Clements L, Ondrake J, Pheiffer T, Simpson A, et al. (2013) A Mechanics-Based Nonrigid Registration Method for Liver Surgery using Sparse Intraoperative Data. IEEE Trans Med Imaging 33:1–12. doi: 10.1109/TMI.2013.2283016. [4] Sultan B, Byrne PJ (2011) Custom-made, 3D, intraoperative surgical guides for nasal reconstruction. Facial Plast Surg Clin North Am 19:647–53, viii–ix. doi:10.1016/j.fsc.2011.07.008. [5] Ahn H-W, Chang Y-J, Kim K-A, Joo S-H, Park Y-G, Park K-H (2014) Measurement of three-dimensional perioral soft tissue changes in dentoalveolar protrusion patients after orthodontic treatment using a structured light scanner. Angle Orthod 84. doi: 10.2319/112913-877.1.

Computerized evaluation of the rib kinetics with vector analysis in dynamic chest radiography H. Matsuda1, R. Tanaka2, S. Sanada2, K. Sakuta3, Y. Kishitani4 1 Kanazawa University, Division of Health Sciences, Graduate School of Medical Sciences, Kanazawa, Japan 2 Kanazawa University, Department of Radiological Technology, School of Health Sciences, College of Medical, Pharmaceutical and Health Sciences, Kanazawa, Japan 3 Kanazawa University Hospital, Radiology, Kanazawa, Japan 4 TOYO Corporation, Tokyo, Japan Keywords Dynamic chest radiography  Rib movement  Vector analysis  Bone suppression technique

Purpose Abnormal motion of the rib cage is very common in patients with chronic obstructive pulmonary diseases (COPD). Therefore, understanding rib kinetics is crucial for the evaluation of pulmonary function and treatment effects. Dynamic chest radiography using a flat-panel detector (FPD) allows the cost-effective evaluation of pulmonary function at low radiation doses. Recent research indicates that vector analysis is a simple and promising approach for the evaluation of rib kinetics; additionally, dynamic bone images obtained using bone suppression image-processing improves the accuracy of the analysis of rib kinematics [1]. However, there are no established diagnostic criteria for rib kinematics on dynamic chest radiographs. The present study was performed to evaluate the normal variation of rib kinetics in order to provide the basis for establishing the diagnostic criteria. Methods Image acquisitions: Dynamic chest radiographs of 129 patients (21–91 years old; mean, 48.5 years; M:F = 89:40) were obtained using a dynamic FPD system with exposure conditions of 110 kV, 80 mA, 6.3 ms, SID of 2.0 m, and 3.0 fps. Imaging was performed during respiration, in the standing position and posteroanterior (PA) direction. The total exposure dose was almost the same as that in conventional chest radiography in two projections (PA + LA). Image analysis: Dynamic chest radiographs were processed using a commercial bone suppression image-processing software to separate the soft-tissue and bone components [2, 3]. The resultant bone images were divided into 1-cm2 blocks, and the velocity vectors in each block were measured in order to estimate the direction and speed of rib movement from one image to another. The velocity vector measured on the body counter was used to detect body motion, which was corrected by shifting the images in the opposite direction of the detected body motion. The velocity vector measured on the ribs was analyzed for symmetry in both hemithoraces and synchronization with the respiratory phase. The symmetry in both of the hemithoraces was assessed based on the vector sum in the horizontal direction, assuming that it should be zero irrespective of the respiratory phase in normal rib kinetics because of the symmetry in moving directions in both hemithoraces. The synchronization with the respiratory phase was assessed based on the correlation coefficient between the diaphragm displacement per frame and vector sum in the vertical direction in each frame. We determined normal rib movement based on the findings in 27 patients who had were confirmed to be normal based on chest radiographs and the results of pulmonary functional test. For a better understanding of the findings of the normal control images, the results were compared to those previously analyzed in cases with lung diseases such as pneumothorax, pneumonia, emphysema, asthma, and lung cancer. Results In many of the normal control images, the direction of the rib movement during inspiration was oblique, upward, and outward, while that during expiration was oblique, downward, and inward (Fig. 1). The vector sum in the horizontal direction was less than 310 mm (average vector per block was 1.0 mm), except for three cases with a slight deviation in left/right rib movement timing (Fig. 2). The diaphragm displacement per frame and vector sum in the vertical direction in each frame were highly correlated (r = 0.84). These results indicated that the ribs moved symmetrically in both the hemithoraces and in synchrony with the respiratory phase. The rib movement in patients with unilateral impairments has been reported to be asymmetrically distributed and less synchronous with the respiratory phase. With the present method, abnormal rib movement could be differentiated from normal movement based on the deviation from uniformity and right/left symmetry of the velocity field on the dynamic bone images.

123

Int J CARS

1

University of Warsaw, Interdisciplinary Centre for Mathematical and Computational Modelling (ICM), Warsaw, Poland 2 Sport Medica, Radiology Department, Warsaw, Poland

Keywords MRI  Haralick’s texture features  Achilles tendon  Image properties

Fig. 1 Velocity vector map of a normal control (32-year-old man)

Fig. 2 Variation of vector sum in the horizontal direction. (a) Normal controls (n = 27), (b) Abnormal cases (n = 6) Conclusion The results of our study confirmed that normal rib movement shows symmetrical distribution and synchrony with the respiratory phase. Our findings could aid the development of diagnostic criteria for detecting abnormal rib kinematics on dynamic chest radiographs. Further studies are required to develop a computerized method for the detection of abnormalities based on the non-uniformity, right/left asymmetry, and phase shifting of the velocity field. References [1] Tanaka R, Sanada S, Sakuta K, et al. Quantitative analysis of rib kinematics based on dynamic chest bone images: preliminary results. Journal of Medical Imaging. 2(2) 024002, 2015 (Online). [2] Suzuki K, Abe H, MacMahon H, Doi K. Image-processing technique for suppressing ribs in chest radiographs by means of massive training artificial neural network (MTANN). IEEE Trans Med Imaging 25:406–416, 2006. [3] Hogeweg L, Sa´nchez CI, van Ginneken B. Suppression of translucent elongated structures: applications in chest radiography. IEEE Transactions on Medical Imaging 2013;32:2099–2113.

Purpose The aim of this work was to find the optimal non-invasive method of the Achilles tendon state assessment by means of Magnetic Resonance Imaging (MR). Achilles tendon is relatively prone to ruptures, however, at the same time it is the most common rupturing one. The problem concerns mainly sport practitioners—both professionals and amateurs. Usually the rupture is caused not by a very strong trauma or overload. The main reason is progressive degeneration, caused by a number of overloads or micro-injuries, metabolic diseases and other medical disorders. The increased load at break is only a trigger. Non-invasive assessment of the tendon is important for two reasons. Firstly, to assess the condition of the tendon in athletes—to determine susceptibility to rupture and to monitor potential pathologies. Secondly, in order to monitor the healing process. The state-of-the-art methods in radiology for tendon assessment are strongly subjective (operator-dependent) and there are no established computer assistance methods and no quantitative descriptors to objectify the diagnostic procedures. Methods In our pilot study, one tendon of healthy volunteer (Control) and one patient tendon after reconstruction surgery have been examined (Patient) for comparison. For both tendons a series of more than 20 MR sequences were performed with different protocols, including experimental approaches with DTI and DWI. After preliminary analyses (e.g. signal-to-noise ratio, information amount, image quality) the list was limited to: PD (proton density) T1_FSE (first spin echo) T2 T2* GRE (gradient echo) T2* GRE TE_MIN (minimal time echo) In Phase Ideal Out Phase Ideal Water Ideal. All imaging measurements were done using magnetic resonance imaging device GE Signa HDxt 1.5T. For further numerical analysis each tendon was manually segmented using the OsiriX software [1] by radiology expert. Calculations were made using VisNow, an open-source visual analysis platform [2]. All analyses were performed in the segmented tendon ROI only. In order to assess the usefulness of particular sequences to provide information on the state of the tendon, and significantly to provide differentiation between healthy and unhealthy one, we focused on two parameters, by analogy with the ANOVA, where ‘‘between-subject’’ and ‘‘within-subject’’ variances are considered. The first analysed parameter was the normalized (with regard to the standard deviations of the signals) difference of averages (NDOA): NDOA ¼ ðavgðdata patientÞ  avgðdata controlÞÞ=  sqrtðstdðdata patientÞ2 þ stdðdata controlÞ2 Þ;

Assessment of the Achilles tendon by Haralick’s texture features of MRI images: preliminary study J. Zielin´ski1, P. Regulski1, N. Kapin´ski1, B. Ciszkowska-Lyson´2, B. A. Borucki1, K. S. Nowin´ski1

123

calculated for all corresponding axial cross-sections (slices) and understood as dissimilarity measure. The second parameter was the ratio of the variance between-slices and the total variance (quotient of variances, QOV):

Int J CARS QOV ¼ variance over slices=total variance: If the QOV is close to 1 it shows large internal differences in the tendon condition along its longitudinal axis. If the QOV is close to 0 it shows internal uniformity. The parameters were derived for all sequences. Additionally, for each of 8 sequences a set of 12 modified Haralick’s texture features [3, 4] have been calculated in local neighbourhoods (5 9 5 9 5 pixels window), providing altogether 104 3D images to compare. Slight modifications of the feature calculation were related to the anisotropy of the MRI spatial resolution. Just as measured sequences, Haralick’s texture features were compared in terms of differentiation between healthy and sick tendon. Results We calculated the difference (NDOA) between the healthy and the injured/reconstructed tendon, as described in Method. The lowest values of parameter NDOA were received for sequences T1 and T2 (NDOA = 0.27 for both of them). It means that the standard MRI scan sequences poorly differentiate between healthy tendon and sick one. Top differentiate sequences are PD (NDOA = 0.59) and Water Ideal (NDOA = 1.01), and the T2* GRE TE_MIN sequence (NDOA = 0.73). This means that for the best sequences differences between the tendons are large compared to the noise and variation of signal within the tendon. For a healthy tendon the QOV was rather small (varied from 0.1 for Water Ideal sequence to 0.32 for T2* GRE). ‘‘Traditional’’ sequences T1_FSE and T2 of Patient tendon were also characterized by a low quotient of variance (0.29 and 0.24 respectively). For other Patient sequences the quotient of variance was significantly higher (the highest for T2* GRE TE_MIN: QOV = 0.88). This analysis leads to information on tendon inhomogeneity along its longitudinal axis. Again, the best results were obtained for T2* GRE TE_MIN sequence. The parameter describing the difference between the tendons (NDOA) achieved much greater values, calculated from most of Haralick’s texture features, as compared to unprocessed MR images (up to NDOA = 2.09 for ‘‘Sum_entropy’’ in T2* GRE TE_MIN sequence), Fig. 1.

Research and Development (Poland) within STRATEGMED programme (STRATEGMED1/233224/10/NCBR/2014). Conclusion The preliminary results obtained for the two compared datasets, together with the calculated NDOA and QOV parameters show promising results for further deeper investigation on larger statistics. First of all, there exists a significant difference in internal tendon uniformity along the longitudinal tendon axis. Accordingly to the expectations, a healthy tendon is more uniform internally than the injured/reconstructed one, what is shown in QOV analysis. The T2* GRE TE_MIN and InPhase Ideal protocols were most descriptive. Secondly, it seems clear based on NDOA results that the Haralick statistics applied to MRI images with proper sequence/protocol provide very sharp differentiation between normal and pathological state of the Achilles tendon as compared to unprocessed images. The best differentiating protocols (original sequences) were T2* GRE TE_MIN and Water Ideal, while the most significant results were obtained for entropy based Haralick’s features in T2* GRE TE_MIN and contrast based Haralick’s features in PD, T1 FSE, T2 and InPhase Ideal. However, Water Ideal difference is directly connected to higher water content (edema) in the sick tendon. A further study with increased amount of data will allow for multivariate analysis of the texture characteristics and allow to correlate the image based indicators with the experrt assessment of the process of the tendon healing. We intend to introduce normalization of imaging data to eliminate crude errors caused by non-uniformity of MRI data and move in the direction of both qualitative and quantitative MRI. However, in this preliminary study we’ve shown that Haralick’s features provide added value to original image analysis and that MR imaging techniques with computer assistance analysis have a promising potential for non-invasive quantitative assessment of tendon pathology. References [1] OsiriX software,http://www.osirix-viewer.com [2] Nowin´ski KS, Borucki B ‘‘VisNow—a Modular, Extensible Visual Analysis Platform’’, 22nd Int. Conf. in Central Europe on Computer Graphics, Visualization and Computer Vision WSCG2014,http://visnow.icm.edu.pl. [3] Haralick RM ‘‘Statistical and structural approaches to texture’’, Proc. of the IEEE, Vol. 67(5), 1979, pp.786–804. [4] Ahuja N, Rosenfeld A, Haralick RM ‘‘Neighbor gray levels as features in pixel classification’’, Pattern Recognition, Vol. 12(4), 1980, pp. 251–260.

A semiautomatic tool for quantitative evaluations in 7T MR (magnetic resonance) images of the brain

Fig. 1 The top row shows the 3D ROI of the Achilles tendon in PD (left) and T2* GRE TE_MIN (right) protocols. The bottom row shows the Haralick’s Varaince for T2* GRE TE_MIN (left) and the 3D localization of the segmented tendon (right) The following work was part of ‘‘Novel Scaffold-based Tissue Engineering Approaches to Healing and Regeneration of Tendons and Ligaments (START)’’ project, co-funded by The National Centre for

M.E. Fantacci1,2, G. Donatelli3, L. Biagi4, M. Costagli4,5, D. Frosini6, A. Giuliano1,2, A. Retico2, G. Tognoni6, M. Cosottini3,5, M. Tosetti4,5 1 University of Pisa, Department of Physics, Pisa, Italy 2 INFN, Pisa Section, Pisa, Italy 3 University of Pisa, Department of Translational Research and New Technologies in Medicine, Pisa, Italy 4 IRCCS Fondazione Stella Maris, Pisa, Italy 5 Fondazione IMAGO7, Pisa, Italy 6 University of Pisa, Department of Clinical and Experimental Medicine, Pisa, Italy Keywords Magnetic resonance imaging  Image processing  Hippocampus  Mild cognitive impairment Purpose Aim of this work is to develop a semi-automated procedure to measure the thickness of the stratum radiatum and lacunosum-moleculare

123

Int J CARS (SRLM) of the hippocampus in 7T high-resolution T2*-weighted MR images of MCI (Mild Cognitive Impairment) patients. The goal consists in investigating the correlations of SRLM thickness with clinical scores (the Mini-Mental State Examination score and the Free and Cued Selective Reminding Test) of the patients. Changes in the SRLM thickness are supposed to have a role in the pathological cognitive decline. Recent studies highlighted the relevance of this subfield in the early stages of the mild Alzheimer’s Disease [1]. MCI, a cognitive decline greater than expected for age and education that can evolve to dementia, is an interesting transitional stage to investigate. MRI at ultra-high magnetic field (C7 Tesla) allows the representation of anatomical structures at sub-millimeter resolution [2], so small structures such as hippocampus subfields become visible. Methods A semi-automatic procedure, which imports some basic ideas of the algorithm presented in [3], has been set up on a sample image of a healthy volunteer and then applied to MCI patients to investigate the relationship between SRLM thickness and clinical scores. An oblique coronal slice prescribed perpendicularly to the axis of the hippocampus, where the SRLM is visible, is presented to the user, who is asked to select the centre of the Region of Interest (ROI) for the following steps. The ROI is zoomed to allow the user to modify the image contrast and brightness and to draw a line by mouse clicking along the curved shape of the SRLM. Then, the line defined following the curve shape of the SRLM is interpolated with a spline (Fig. 1); the normal directions to the spline are computed and the normal vectors are overlaid to the original ROI and shown; the image intensity profiles along the normal directions are computed and mounted in 2D image, where the hippocampus appears as unrolled along the SRLM; the SRLM appears as a dark straight band in this 2D image, which is finally squeezed along the SRLM direction to obtain the average of the image intensity across all normal profiles; as this averaged intensity profile shows a Gaussian shape, a fit with a Gaussian function is carried out, and its width is retained as a measure of the SRLM thickness. To check the reproducibility of this measure, one of the user of the tool was asked to repeat ten times the same measurement for both left and right hippocampi of one subject. Once the algorithm has been developed and validated on the 7T MR image of the healthy volunteer, it has been used in a clinical study involving subjects affected by MCI. Ten MCI patients underwent a brain 7TMR examination including a high-resolution 2D T2*-weighted sequence targeting hippocampus. An experienced neuroradiologist used the semi-automatic image processing tool we developed to delineate the SRLM profile on a coronal oblique slice of the 7T T2*weighted MRI of each MCI patient. The thickness of the SRLM was estimated for both the right and the left hippocampi. The patients underwent a neuropsychological battery including the Mini-Mental State Examination (MMSE) and the Free and Cued Selective Reminding Test (giving great attention to the free recall FCSRT-FR) and the results have been correlated with the thicknesses. MR images were acquired with the 7T MR950 scanner (GE Healthcare Medical Systems, Milwaukee, WI, USA) of the IMAGO7 Foundation in Pisa (Italy), equipped with a 2ch-Tx/32ch-Rx head coil (Nova Medical, Wilmington, MA USA). The acquisition protocol included a highresolution 2D T2*-weighted GRE (gradient-recalled echo, TE = 22 ms, TR = 240 ms, in-plane resolution 0.3 9 0.3mm2, slice thickness = 2 mm) sequence prescribed perpendicular to the longitudinal axis of the hippocampus. The algorithm has been implemented in Matlab (R2009b, The MathWorks, Inc.), and its execution is managed by a dedicated GUI.

123

Fig. 1 Left: the ROI is enlarged to allow the user draw the SRLM profile; Right: the points are interpolated with a spline and the normal vectors to the profiles are computed Results In the test of reproducibility, we obtained the following measures of the SRLM (average ± standard deviation [range of values]): (1.61 ± 0.10) mm [1.51–1.83 mm] for the left hippocampus and (1.53 ± 0.04) mm [1.47–1.61 mm] for the right hippocampus. The measure has been considered as highly reproducible, as the estimated error is limited to few per cents of the measured thickness, i.e. the 6 % and the 3 % for the left and right hippocampus, respectively. In MCI patients, the correlation between the measured SRLM thickness and the numerical scores provided by the neuropsychological tests were analysed according to the Spearman’s rank test. Are preliminarly shown the average SRLM thickness (average value between the left and right hippocampi) correlated with MMSE score (r = 0.60; p \ 0.1; n = 10) and the average SRLM thickness of the right hippocampus correlated with the FCSRT-FR (r = 0.97; p \ 0.05; n = 5). Conclusion We propose a semiautomatic image processing tool to measure the thickness of small structures in anatomical images. As a case study we focused on the measure of one of the hippocampal subfields, the SRLM, which is supposed to be involved in early degenerative changes related to pathological cognitive decline. We found that the SRLM thickness is correlated with the MMSE and the FCSRT-FR neuropsychological scores. The 2D nature of the procedure can be extended to 3D data, where available with the appropriate spatial resolution. As high-resolution structural imaging that can be acquired with MRI at UHF (7T) allows the visualization of very thin anatomical structures, suitable tools to measure their size in a reproducible way might have a fundamental role in clinical research studies. References [1] Brown TI, Hasselmo ME et al. (2014) A high-resolution study of hippocampal and medial temporal lobe correlates of spatial context and prospective overlapping route memory. Hippocampus 24(7)7: 819–839. [2] Thomas BP, Welch EB et al. (2008) High-Resolution 7T MRI of the Human Hippocampus In Vivo. J MRI 28: 266–1272. [3] Kerchner GA, Deutsch et al. (2012) Hippocampal CA1 apical neuropil atrophy and memory performance in Alzheimer’s disease. Neuroimage 63: 194–202.

Int J CARS Wavelength-swept laser based wavelength scanning interferometry for 3D surface measurement M. F. Shirazi1, R. E. Wijesinghe1, K. Park1, P. Kim2, M. Jeon1, J. Kim1,3 1 Kyungpook National University, School of ELectronics Engineering, Daegu, South Korea 2 Oz-tec Co., Ltd., Daegu, South Korea 3 Kyungpook National University, Daegu, South Korea Keywords Optical  Wavelength scanning  Interferometer  Laser Purpose Wavelength scanning interferometry with galvo filter based wavelength-swept laser is employed for 3D surface measurement. Using wavelength-swept laser, area scanning is done in spectral domain without any mechanical scanner. The proposed system has large field of view and can be helpful in real time scanning. Methods Figure 1 shows the experimental setup of wavelength-swept laser [1] based wavelength scanning interferometer. The system consists of Michelson interferometer with broadband wavelength-swept laser with 1 Hz repetition rate and a near infrared CMOS camera with a frame rate of 340 fps. The near infrared light split by beam splitter and divided into sample and reference arms. When the two arms are matched together then the interference signal is captured by camera. The Fourier transforms of acquired raw signal after zero padding and Gaussian windowing gives the intensity and phase information. The signal processing of this raw signal provides the measurement in micrometer to nanometer range. Graphical processing unit (GPU) is used for fast signal processing of acquired signals. The system is calibrated using point spread function measurement with micrometer stage. The standard step height target, reflective and refractive samples are utilized to demonstrate the performance of system.

Fig. 2 Wavelength scanning interferometry result. (a) 500 Korean won coin. (b) The surface reconstruction using wavelength scanning interferometry Conclusion The proposed system can be used for 3D reconstruction of image with micrometer accuracy in real time. This system can be employed for high resolution volumetric imaging. The proposed system can be integrated with commercial clinical surgical microscope for virtual intraoperative surgery [2] and similar applications. References [1] Shirazi MF, Jeon M, Kim J ‘‘850 nm centered wavelengthswept laser based on a wavelength selection galvo filter,’’ Chinese Optics Letters, vol. 14, p. 011401, Jan 2016. [2] Lee C, Kim K, Han S, Kim S, Lee JH, Kim HK, Kim C, Jung W, Kim J ‘‘Stimulated penetrating keratoplasty using real-time virtual intraoperative surgical optical coherence tomography,’’ Journal of Biomedical Optics, vol. 19, p. 30502, Mar 2014.

Towards classification of human hepatic and malign tissue using diffuse reflectance spectroscopy and impedance spectroscopy A. Keller1, T. Wilhelm2, M. Vetter1 1 Mannheim University of Applied Sciences, EMB-Lab, Mannheim, Germany 2 University Medical Centre Mannheim, Surgical Department, Mannheim, Germany Keywords Diffuse reflectance spectroscopy  Impedance spectroscopy  Biopsy needle  Tissue differentiation Fig. 1 Experimental setup of wavelength scanning interferometry. CMOS: complementary metal oxide semiconductor, WSL: wavelength swept laser Results A step height standard target is utilized to verify the performance of system. A portion on metallic coin, glass coverslip and micro lens arrays are reconstructed using proposed system. Depending on the sample characteristics, the topographical and tomographical information of the sample can be extracted from 3D reconstructed image. Figure 2 shows the reconstructed image from acquired raw data of 500 Korean won coin. The coin portion in black box as shown in Fig. 2(a) is reconstructed using wavelength scanning interferometry. The enface reconstructed portion in Fig. 2(b) shows the topological variation as correlated with the coin.

Purpose Biopsies of soft tissues are being intensively discussed because of the risk of tumor cell seeding. This procedure can cause the formation of subcutaneous metastases in the needle tract or distant metastases via the interstitial fluid. Since 90 % of the cancer mortality is caused by metastases, every biopsy is critically reviewed for the necessity [1]. Furthermore there is often a radiation exposure induced by control scans during the biopsy procedure. Despite these complications the biopsy is considered to be the standard for the diagnosis of hepatic cancer due to its accuracy. A study concluded that 21 % of biopsies of hepatic tissues have to be repeated at least once, because the amount of cells or tissue volume is insufficient or the needle tip is positioned improperly [2]. In this case it is necessary to repeat the biopsy procedure which involves a higher risk of bleeding and tumor cell seeding.

123

Int J CARS By using in vivo tissue differentiation, implemented in a common biopsy needle, a better localization of tumor tissue can be achieved, especially of the more informative tumor boundary. Consequently it is likely that the amount of biopsy procedures and the radiation exposure induced by control scans can be reduced. Such an ‘‘intelligent biopsy needle’’ will provide additional information with the detection of tissue boundaries. Following literature review and preliminary tests with ex vivo tissue of pigs we consider the diffuse reflectance spectroscopy (DRS) in the visible and near infrared spectral range and the impedance spectroscopy (IS) as promising methods to improve the diagnosis of cancer. A clinical pilot study was started to evaluate the ability of DRS and IS to discriminate healthy hepatic tissue and tumor tissue of 30 different patients. Methods The study was conducted in collaboration with the surgical clinic of the University Medical Centre Mannheim, under approval of the protocol and the ethics review board. In this preliminary analysis liver tissue was obtained from 18 patients undergoing a segmental liver resection for primary liver cancer or liver metastases from colorectal origin. Ex-vivo DRS-spectra were acquired using a modular USB spectrometer (450 nm-900 nm) with a Tungsten halogen broadband light source. For the IS a conventional biopsy needle was modified based on the idea of Mishra et al. [3] to measure the electrical impedance of tissues surrounding the needle tip. With a programmable LCR-Bridge the measurements were performed at 40 different frequencies between 10 Hz and 200 kHz. Within half an hour after the surgical resection, the tissue was inspected by the surgeon and healthy liver and malign lesions were identified. Incisions in both tissue classes were made and multiple measuring points in both tissues defined. Then the measurements with DRS and IS were performed at the measuring points. A total of 161 (18 patients) DRS and 158 (17 patients) IS measurements were acquired. The averaged measurements of each patient and each tissue class were used to classify the data, because the data of each patient is statistically correlated. Each DRS measurement was normalized by the maximum intensity of the spectrum to remove the variation in each class. The results of the IS measurements in real and imaginary part were transformed to the magnitude and angle of the spectra normalized by the mean of each spectrum. This normalization compensates systematic changes in the spectrum but the spectral shape remains unchanged. A principle component analysis (PCA) was performed on the entire dataset of both methods to reduce the dimensionality for the classification. The first two principle components (PC) were used to classify the measurements into healthy tissue or malign lesions. For the validation of the trained classification model the leave-one-out cross validation was applied to split the data into a training- and a test-dataset. Each prediction of a model was compared to the reference information of the measurement and used to evaluate the recognition rate of the classification. In this preliminary analysis 18 patients were included, mean age was 63 ± 14 years (range 37–83 years), whereas 14 participants were male. Five patients had primary liver cancer and 13 patients had liver metastases from colorectal cancer. Impedance measurements of one patient were excluded, because the correct positioning of the probe was not guaranteed. Figure 1 represents the preprocessed spectra of DRS (A) and the angle of IS (C) measurements after normalization at healthy hepatic tissues (blue) and malign lesions (red). Especially the DRS data indicates that a classification which distinguishes between the examined tissues seams feasible. Figure 1(B) shows the score values of the first two PCs calculated from DRS spectra which explain a variance of 96 % of the data. The score values of the different classes

123

are grouped at the positive and negative range of the two axes. The linear discriminant analysis (LDA) was used to classify the score values. The score plot of the impedance spectra also represents a grouping of the tissue classes, as seen in Fig. 1(D). About 99 % of the variance can be explained with the first two PCs of the PCA. A Naı¨ve Bayes classifier was used to classify the score values of IS measurements.

Fig. 1 Diffuse reflectance measurements normalized by maximum intensity (A) Score plot of the first two principle components calculated from the normalized DRS spectra (B) Angle of impedance measurements normalized by mean value (C) Score plot of the first two principle components calculated from the normalized IS spectra (D) Results The classification of liver and tumor tissues with the DRS measurements achieved an accuracy of 100 % using LDA. The classification of the two tissue classes with the IS achieved also an accuracy of 88 % using Naı¨ve Bayes classifier. Conclusion Based on this preliminary study it can be expected, that the DRS in the visible and near infrared spectral range and the IS are reliable methods to distinguish between healthy and malign liver tissues. This hypothesis needs to be confirmed with the patients left to finish the clinical study. References [1] Li J, King MR (2012) Adhesion receptors as therapeutic targets for circulating tumor cells. Front Oncol 2:79. ¨ ber die Wertigkeit Ultraschall-gesteuerter [2] Thielke S (2014) U Punktionen von unklaren hepatischen Raumforderungen. Medizinische Fakulta¨t der Universita¨t Go¨ttingen, Diss. [3] Mishra V, Bouayad H, Sched A, Hartov A, Heaney J, Halter RJ (2012) A real-time electrical impedance sensing biopsy needle. IEEE Trans Biomed Eng 59(12): 3327–3336.

PACS in a suitcase: a cost effective open solution for emerging countries—the Bhutan project O. Ratib1, N. Roduit1, D. Nidup1, A. Geissbuhler1 1 University Hospital of Geneva, Radiology, Geneva, Switzerland Keywords PACS  Open source  OsiriX  Weasis

Int J CARS Purpose Digital imaging modalities are becoming more widely available in hospitals and healthcare facilities of emerging countries providing remote areas with higher quality of care and diagnostic capabilities for patient management. While most of these centers are equipped with state-of-the-art imaging equipment, they often lack the needed IT infrastructures to manage and distribute the digital imaging data (Figs. 1, 2).

Fig. 1 Diagram of the different imaging modalities connected to the PACS server and the distribution to OsiriX workstations for diagnostic interpretation and to clinical wards through Weasis, a web-based viewer

Fig. 2 The radiology reading room that was installed with two OsiriX workstations for diagnostic interpretation by radiologists at the Hospital of Thimphu in Bhutan The goal of our project was to design and implement a robust, compact and cost-effective picture archiving and communication system (PACS) suitable for radiology centers in small hospitals. It was implemented in the main reference hospital of Bhutan equipped with a CT, an MRI, a digital radiology and a suite of several ultrasound units. Until now this hospital did not have any informatics infrastructure for image archiving and interpretation and needed a system for distribution of images to clinical wards. Methods In Thimphu, the capital Bhutan, a small kingdom of little over 700’000 inhabitant south of the chain of the Himalayan, the Jigme Dorji Wangchuck National Referral Hospital is the only radiology center equipped with a CT scanner, and MRI scanner, digital radiology unit and a suite of 5 ultrasound units.

All these modalities have the ability to export images digitally in DICOM compliant format however they are not connected to any archiving system. The hospital is equipped with a secured Ethernet local area network available across the hospital as well as a restricted Wi-Fi network. The radiology department benefits from a separate sub-network allowing higher-bandwidth communication bandwidth between different imaging equipment. The solution developed for this project combines several OpenSource software platforms [1] in a robust and versatile archiving and communication system connected to analysis workstations equipped with FDA-certified OsiriX software a professional version of the highly popular Open-Source OsiriX software [2, 3]. The whole system was implemented on standard off-the-shelf single computer unit. Results The system was installed in 3 days by connecting all imaging modalities of the radiology department. Training of the radiologists as well as the technical and IT staff was provided on site to ensure full ownership of the system by the local team. Radiologist were rapidly capable of reading and interpreting studies on the OsiriX workstations which had a significant benefit on their workflow and ability to perform diagnostic tasks more efficiently. The server also allowed providing access to the images to several clinical units on standard desktop computers through a web-based viewer. Conclusion This pilot project of implementing a robust and cost-effective PACS system for image archiving and distribution allowed us to establish a model of a system that can easily be disseminated in large number of institutions around the world that already have digital imaging modalities but cannot afford an Image management infrastructure or commercial PACS systems [4]. The PACS server architecture that we developed based on existing open source components can easily be replicated on standard off-the-shelf hardware [5]. The global package is made available as free Open Source software by us on the publicly accessible server (https://github.com/nroduit/openmediavault-dcm 4chee). References [1] Ratib O, Rosset A, Heuberger J (2011) ‘‘Open Source software and social networks: disruptive alternatives for medical imaging.’’ Eur J Radiol 78(2): 259–265. [2] Valeri G, Mazza FA, Maggi S, Aramini D, La Riccia L, Mazzoni G, Giovagnoni A (2015) ‘‘Open source software in a practical approach for post processing of radiologic images.’’ Radiol Med 120(3): 309–323. [3] Yamauchi T, Yamazaki M, Okawa A, Furuya T, Hayashi K, Sakuma T, Takahashi H, Yanagawa N, Koda M (2010) ‘‘Efficacy and reliability of highly functional open source DICOM software (OsiriX) in spine surgery.’’ J Clin Neurosci 17(6): 756–759. [4] Rosset C, Rosset A, Ratib O (2005) ‘‘General consumer communication tools for improved image management and communication in medicine.’’ J Digit Imaging 18(4): 270–279. [5] Valente F, Silva LaB, Godinho TM, Costa C (2015) ‘‘Anatomy of an Extensible Open Source PACS.’’ J Digit Imaging. (Online publication prior to print)

Response to liver radioembolization predicted by quantitative SPECT/CT of 99mTc-macroaggregated albumin (MAA) and 90Y microsphere distribution N. Hall1,2, J. Zhang2, V. Nagar2, M. Robertson3, C. Wright2, H. K. Khabiri2, E. Wuthrick2, M. Knopp2 1 Hospital of the University of Pennsylvania, Philadelphia, PA, United States

123

Int J CARS 2

The Ohio State University, Columbus, OH, United States Nationwide Children’s Hospital, Columbus, OH, United States

3

Keywords SPECT/CT  Y-90  Radio-embolization  Therapy Purpose Selective 90Y radioembolization (RE) therapy for hepatic malignancies and metastases involves pre-planning angiography and 99mTc macroaggregated albumin (MAA) scintigraphy to assess hepatopulmonary shunt fraction. If eligible, patients receive either resin or glass microspheres containing 90Y. Due to the absence of discrete gamma photon peaks for 90Y, routine post-RE imaging detects bremsstrahlung radiation produced by 90Y decay for qualitative assessment. The purpose of this pilot study was to determine whether new clinical quantitative SPECT/CT imaging approaches can be used as a prognostic indicator during initial evaluation and subsequent treatment of patients with RE. Methods 22 patients with primary hepatic malignancies underwent pre-RE angiography, pre-RE MAA planar and SPECT/CT imaging, and postRE 90Y bremsstrahlung planar and SPECT/CT imaging. Quantitative assessment of tumor activity MAA and 90Y bremsstrahlung SPECT/ CT were correlated with each other as well as post-RE imaging response and progression-free survival. For each study, a single representative lesion was first chosen on the pre-therapy contrast enhanced diagnostic CT scan (n = 21) or MRI (n = 1) and ROI defined. ROIs were also selected for measuring background activity in the disease-free liver parenchyma and the arterial distribution for the target lesion intended for radioembolization. Tumor-to-background ratio (TBR) was calculated using maximum counts per voxel within the tumor volume of interest (VOI) based on diagnostic imaging and mean counts per voxel for background normal liver parenchyma. TBRs measured on MAA and 90Y studies were compared. Sensitivity and specificity for prediction of response were then calculated based on MAA TBR. Results MAA Overall SPECT/CT-derived TBR values for target lesions on the MAA studies were 8.0 ± 11.6 (range 1.3–55.2; n = 22). For CRC group, TBR was 7.8 ± 14.4 (range 1.6–55.2; n = 13). Non-CRC metastases had a TBR of 8.3 ± 6.6 (range 1.3–22.1; n = 9). There was no statistical difference in hepatopulmonary shunt fraction between the two groups with 7 ± 3 % (range 2.6–12.9 %) for the CRC group and 6.5 ± 2.7 % (range 3.3–12.4 %) for the non-CRC group. The mean MAA tumor-to-background ratio (TBR) in those patients whose disease progressed despite RE was 2.7 ± 1.3 (n = 8) while the mean MAA TBR in the patients whose disease did not progress was significantly higher at 11.0 ± 13.8 (n = 14, P \ 0.05). Using a threshold cutoff MAA TBR of 2.5, the overall sensitivity and specificity were 63 % and 93 %, respectively, in differentiating patients who were likely to demonstrate non-progression following RE as oppose to those patients who would likely progress despite RE. 90 Y SPECT/CT-derived 90Y TBR for target lesions on the 90Y bremsstrahlung imaging measured on the post therapy 90Y studies had a 90Y TBR of 3.0 ± 2.5 (range 1–11; n = 22). For the CRC group, the 90Y TBR was 2.5 ± 2.6 (range 1 -11; n = 13). For the non CRC group, the 90Y TBR had a mean of 3.7 ± 2.2 (range 1.3–8.2; n = 9). Overall, there was good correlation between the TBRs measured on pre-therapy MAA and post therapy 90Y scans (r = 0.82). This correlation was observed to be particularly stronger in those with CRC metastases (r = 0.98) than for those in the non-CRC group (r = 0.69). Conclusion Quantitative SPECT/CT assessment indicates that pre-RE intratumoral MAA activity is an important predictor for subsequent response

123

to RE for both CRC and non-CRC. Continued improvement of computer-assisted techniques in pre-therapy quantitative SPECT/CT imaging post processing could better improve subsequent minimallyinvasive radiotherapeutic interventions and selection of patients most likely to respond to therapy. Higher MAA HAP and 90Y TBR are both associated with nonprogression of disease as determined by follow-up imaging. Having absent activity of the MAA HAP study could be used to select patients who will not benefit from RE and should be subjected to this expensive and invasive therapy. Using a MAA TBR cutoff of 2.5 combined with pattern of MAA HAP uptake to predict progression of disease may be useful in counseling patients on their choice to undergo the RE procedure.

Clinical experiences using Digital PET/CT: leveraging innovative technologies and managing disruptive procedural changes in PET imaging M. V. Knopp1, K. Binzel1, C. Wright1, M. I. Knopp1, P. Bardos1, C.-H. Tung2, J. Zhang1 1 The Ohio State University Wexner Medical Center, Wright Center of Innovation in Biomedical Imaging, Columbus, United States 2 Philips Healthcare, Cleveland, United States Keywords Digital PET/CT  Digital photon counting  Precision medicine  High-definition imaging Purpose Solid state, digital photon counting PET/CT (dPET) became available for clinical evaluation using commercial system technology with a pre-release system in 10/2014. Subsequently we started a series of Phase I to Phase III clinical trials, to assess, compare and validate clinical imaging characteristics. Within these efforts, it became clear that several disruptive clinical PET imaging procedural changes have the opportunity to advance the field of clinical molecular imaging, substantial reducing ionizing dose and/or acquisition time and moving from standard definition image reconstruction to high and ultra-high definition, all disruptive changes to the established clinical practice of PET imaging. In this analysis we present what aspects were leveraged and how we propose that these changes could be clinically managed. Methods A next-generation, best in class time-of-flight resolution, digital PET/ CT system (Vereos, Philips) became available for initial Phase I trials Q4/2014. To enable prospective comparison to current photomultiplier tube-based detector systems, the trial included intra-individual comparison studies performed using the Gemini TF 64 (Philips) PET/ CT (cPET) or similar systems. The dPET system includes many technological advances including a best in class timing resolution of 325 ps vs. 550 ps for time of flight. Early assessment of dPET studies indicated that the technology innovations can only be clinically leveraged, if disruptive changes to current clinical procedures in PET imaging would be implemented. We categorized these changes into 4 buckets: Voxel volume, Reconstruction methodologies, Tracer dose, and Acquisition time. We further paid close attention to correctly label changes in order to avoid ambiguities or misnomers. Therefore, we combine for example several aspects of voxel size changes and reconstruction into broader labels such as standard, high- and ultrahigh definition imaging. Results Disruptive changes are practically unwelcome in clinical imaging as they create conflict with best practice guidelines, clinical customs, clinical experience and many other aspects especially in today’s increasing process, expectation and compliance driven healthcare environment. Accepting this as reality requires new ways of clinical technology introduction. Therefore, intra-individual comparison studies are essential to efficiently generate the objective data. This

Int J CARS well established and validated, preferable and statistically also more powerful approach, now needs to be consistently implement also in the sub-component analysis when further changes (4 buckets defined above) are investigated. While such rigorous approaches combined with refined simulation methodologies generate huge data volumes, they also create the basis for validation and training. We found that this is essential to build the experience and confidence of clinicians to embrace substantial procedural changes. The most effective approach is when the ‘‘before’’, meaning prior procedure quality can be provided alongside the ‘‘after’’, meaning new quality. PET systems with sufficient computational power can readily enable such transition processes, which we feel are enabling and essential for the management of disruptive procedural changes. Conclusion Disruptive clinical procedural changes are increasingly difficult to implement in medical imaging. The introduction of the next generation digital PET technology can serve as a case study, that clinical and methodology validation trials require intra-individual comparisons from current to new procedures. When technology advances can be managed by advanced computational approaches in such a way that new technology enables the simulation of the old/current practice alongside the new disruptive procedure, clinical implementation and adoption can be substantially accelerated and simplified.

Comparing risk estimates following diagnostic CT radiation exposures employing different methodological approaches V. Kashcheev1, V. Ivanov1, A. Menyajlo1 1 A. Tsyb Medical Radiological Research Center, National Radiation Epidemiological Registry, Obninsk, Russian Federation Keywords Computed tomography  Dose equivalent  Effective  Health effects  Radiation risk Purpose The IAEA’s Basic Safety Standards of 2011 require registrants and licensees to ensure that the patient or the patient’s legal authorized representative has been informed, as appropriate, of the expected diagnostic or therapeutic benefits of the radiological procedure as well as the radiation risks. The same requirement is present in the Russian Basic Sanitary Radiation Protection Standards of 2009 [1] and in the Basic Sanitary Rules for Radiation Safety of 2010 [2]. It is the first time the requirements of estimating risk of late stochastic effects associated with medical radiologic exposure at the planning stage have been included in international and national regulatory documents. The impetus for the requirements was the increasing use of radiologic examinations, especially pediatric computed tomography (CT). ICRP Publication 103, recommends evaluating radiation risks associated with diagnostic imaging using doses to the individual tissues at risk. Effective dose can be used for comparing doses from similar diagnostic technologies and procedures in different medical clinics and countries, as well as for comparing different technologies for the same examination if reference patient groups are similar by age and sex [3]. The purpose of the current study is to determine the method of calculating organ doses and cancer risk using dose-length product (DLP) for typical routine CT examinations. The LAR estimated using DLP data was compared with risks calculated with the use of organ doses measured with silicon photodiode dosimeters, which were implanted at various tissue and organ positions within adult anthropomorphic phantoms Methods In our study cancer risks associated with computed tomography are estimated using the model described in ICRP Publication 103. The model allows estimating EAR as a function of sex, age at exposure, attained age and radiation dose for all solid cancers and for a particular tumor. Knowing EAR one can estimate lifetime

attributable risk (LAR) of cancer development following single exposure. According to the ICRP models, LAR is a weighted average obtained with the additive (based on excess absolute risk—EAR) and multiplicative (based on excess relative risk—ERR) models. For estimating EAR and ERR the mathematical models as function of sex, exposure and attained age were used. In the presented study we computed proportion factor for three anatomical regions: chest, abdomen and head. The CT-Expo v2.1 computer program was used to compute values of organ doses and proportion factor for 10 selected commercial scanners: Siemens (Emotion 6, Emotion Duo, Sensation 16, Sensation 64), General Electric (LightSpeed 16, LightSpeed VCT), Philips (Briliance 16, Briliance 64), Toshiba (Aquilion 64, Aquilion Premium). Dosimetry data, based on Monte Carlo methods, were generated for an adult male (ADAM; 170-cm height and 70-kg weight) and for an adult female (EVA; 160-cm height and 60-kg weight) [4]. Results Using the CT-Expo v2.1 computer program, we estimated dose proportionality factors. Average estimated dose-proportionality factors for Chest Routine, Abdominal Routine and Head Routine CT examinations with standard deviations, characterizing distribution of organ-specific values. Using average values of dose proportionality factors to a particular organ, we estimated organ- and tissue-specific equivalent doses to organs and tissues for DLP = 100 mGy cm-1 in case of Chest Routine, Abdominal Routine and Head Routine examinations. Using proposed methodology for converting DLP to an organ dose we estimated as an example lifetime attributable risks, LAR, of cancer development following single exposure for women 30 years old. These estimates were compared with risks calculated with the use of organ doses measured with small (\ 7 mm wide) silicon photodiode dosimeters (34 in total), which were implanted at various tissue and organ positions within adult anthropomorphic phantoms. These values of organ doses of Chest CT examination were taken from the publication of Fujii et al. [5]. The lifetime attributable risks of cancer estimated with organ doses calculated from DLP are compare with the risks estimated for measured with silicon photodiode dosimeters organ doses. The difference is less than 29 %. Conclusion Last years we have seen the dynamic increasing of the numbers of CT procedures. The estimating of radiation risks and benefits as well as the Risk Communication in the cases of medical exposure are necessary. The difference between risks which have been evaluated on the basis of effective doses and organ doses depends on sex, age and scan regions. How important are the differences in risk estimates obtained by using organ versus effective dose? In communicating risk and benefits of a medical procedure to a patient, it is likely that the patient would accept differences of up to 30 % when the absolute risk is very low (on the order of 1/2000). References [1] Standards of Radiological Safety (NRB-99/2009) 2009 Sanitary And Epidemiological Regulations And Standards (Moscow: Federal Centre of Hygiene and Epidemiology of Rospotrebnadzor) (in Russian). [2] Basic Sanitary Regulations for Radiation Safety Assurance (OSPORB-99/2010). Sanitary Regulations, SP2.6.1.26.12-10. Moscow: Centre of Sanitary and Epidemiological Standardization, Hygienic Certification of Russian Ministry of Health; 2010 (in Russian). [3] International Commission on Radiological Protection. The 2007 recommendations of the International Commission on Radiological Protection. ICRP Publication 103; Annals of the ICRP, 37(2–4); 2007a. [4] Stamm G, Nagel HD. CT-expo: A novel program for dose evaluation in CT. Rofo 174:1570–1576; 2002 (in German).

123

Int J CARS [5]

Fujii K, Aoyama T, Yamauchi-Kawaura C, Koyama S, Yamauchi M, Ko S, Akahane K, Nishizawa K. Radiation dose evaluation in 64slice CT examinations with adults and paediatric anthrepomorphic phantoms. Br J Radiol 82:1010–1018; 2009.

Comparison of different reconstruction algorithms for decreasing the exposure dose during digital breast tomosynthesis: A phantom study T. Gomi1, K. Fujita2, M. Goto1, Y. Watanabe1, T. Takeda1, T. Umeda1, A. Okawa3 1 Kitasato University, School of Allied Health Sciences, Sagamihara, Japan 2 National Hospital Organization Tokyo National Hospital, radiology, Tokyo, Japan 3 Nagoya University, Graduate School of Medicine, Nagoya, Japan Keywords Digital breast tomosynthesis  Radiation dose  Reconstruction  Breast imaging Purpose Digital breast tomosynthesis (DBT) is a promising technique for improving early detection rates of breast cancer because it can provide three-dimensional (3D) structural information by reconstructing an entire image volume from a sequence of projection-view mammograms acquired at a small number of projection angles over a limited angular range; the total radiation dose is comparable with that used during conventional mammography screening. DBT has been shown to decrease the camouflaging effect of the overlapping fibroglandular breast tissue, thereby improving the conspicuity of subtle lesions. Several digital mammography-based DBT systems have been developed, and preliminary clinical studies are under way [1]. Wu et al. evaluated the conventional reconstruction algorithm (filtered backprojection: FBP) and statistical iterative reconstruction (IR) algorithm (maximum likelihood expectation maximization: MLEM). The author concluded of MLEM algorithm provided a good balance of image quality between the low and high frequency features. In this study, we chose to focus on the statistical IR technique (MLEM) in addition to the algebraic IR technique (simultaneous iterative reconstruction technique: SIRT). We evaluated and compared the characteristics of the reconstructed images and the possible reduction in the radiation dose associated with MLEM, and SIRT algorithms. Methods The DBT system (Selenia Dimensions; Hologic Inc., Bedford, MA, USA) comprised an X-ray tube with a 0.3-mm focal spot (tube target: W, filtration: 0.7-mm aluminum-equivalent) and a 240 ´ 290-mm digital flat-panel amorphous selenium detector. Each detector element was 70 9 70 lm in size. Tomography was performed using a linear tomographic movement, with a total acquisition time 3.7 s and an acquisition angle of 15. Projection images were sampled during a single tomographic pass (15 projections) and were used to reconstruct tomograms of a desired height. A BR3D phantom (Model 020; CIRS Inc., Norfolk, VA, USA) consists of multiple heterogeneous slabs that mimic the glandular and adipose tissue composition and parenchymal patterns of a human breast. The slabs are made of epoxy resins with X-ray attenuation properties corresponding to 50 % glandular/50 % adipose breast tissue. We arranged the nontarget slabs at the top (20–50 mm) and bottom of the target slab (10 mm). Each radiation dose setup was implemented using the following settings: a reference radiation dose [automatic exposure control (AEC) = the exposure condition at 40-mm thickness and determined tube voltage and tube current values] of 28 kVp, 50 mA; a halfradiation dose of 28 kVp, 24 mA; and a quarter-radiation dose of 28

123

kVp, 12 mA. All target and filter combinations contained tungsten (W) and rhodium (Rh). The three algorithms were implemented using a DBT system and experimentally evaluated using measurements, such as signal difference-to-noise ratio (SDNR) and intensity profile, on a BR3D phantom (in-focus plane image). The possible radiation dose reduction, contrast improvement, and artifact reduction in DBT were evaluated using different exposure levels and the three reconstruction techniques. We performed statistical analysis (one-way analysis of variance) of the SDNR data. Results The effectiveness of each technique for enhancing the visibility of the BR3D phantom was quantified with regard to SDNR (FBP versus MLEM, P \ 0.05; FBP vs. SIRT, P \ 0.05; MLEM vs. SIRT, P = 0.945); the artifact reduction was quantified with regard to the intensity profile. MLEM and SIRT produced reconstructed images with SDNR values indicative of low-contrast visibility. The SDNR value for the half-radiation dose MLEM and SIRT images was close to that of the FBP reference radiation dose image. Artifacts were decreased in the MLEM and SIRT images (in the in-focus plane) according to the intensity profiles that we obtained (Fig. 1).

Fig. 1 Areas of measurement of the SDNR metrics (the image shown: a reconstructed image of the BR3D phantom [in-focus plane]). Comparison of the SDNR values obtained using DBT in the in-focus plane [region of interest (ROI)-1, 6.3 mm u; ROI-2, 4.7 mm u; spheroidal masses (epoxy resin)] for different values of phantom thickness and radiation exposure Conclusion Our empirical results demonstrate that MLEM and SIRT can be used to improve image contrast by suppressing streak artifacts in DBT images obtained using both reference and half-radiation doses. With MLEM and SIRT, the radiation levels may be decreased by 50 % relative to the FBP technique. IR may yield improvements in image quality and a reduction in the radiation dose in comparison with the conventional FBP technique. References [1] Skaane P, Bandos AI, Gullien R, Eben EB, Ekseth U, Haakenaasen U, Izadi M, Jebsen IN, Jahr G, Krager M, Niklason LT, Hofvind S, Gur D. (2013) Comparison of digital mammography alone and digital mammography plus tomosynthesis in a population-based screening program. Radiology, 267, 47–56.

Micro-calcification detection from synthetic mammograms

method

using

adaboost

S.-H. Chae1, J.-W. Jeong1, S. Lee1, E. Y. Chae2, H. H. Kim2, Y.-W. Choi3

Int J CARS 1

ETRI, Daejeon, South Korea Asan Medical Center, College of Medicine, University of Ulsan, Seoul, South Korea 3 KERI, Ansan, South Korea 2

Keywords Digital Breast Tomosynthesis  Computer-aided detection  Micro-calcification detection  Adaboost Purpose Full-Field Digital Mammography (FFDM) is a general diagnosis method for breast cancer screening. However, FFDM has a problem that detection sensitivity of breast cancer on dense breast images. Recently, Digital Breast Tomosynthesis (DBT) reconstructing 3D volume data from several x-ray projection images has been considered as a promising image modality for detection and diagnosis of breast cancer. Also, Micro-Calcifications (MCs) detection methods have been investigated for DBT [1]. MCs, particles of calcium, in breast present an important sign of early breast cancer. Since MCs in DBT are expressed by dividing into several slices, FFDM is more effective to detect MCs than DBT. However, scanning DBT and FFDM respectively demands high-dose of something. Consequently we have conducted research on methods to generate synthetic mammmograms (SMs) from DBT and identifying MCs for SMs. Methods SMs are 2D images generated from 3D volume data. To generate 2D synthetic images we used an average method to reduce noise and MIP to enhance edge information [2]. The MCs detection stage applied in this study will be explained later in detail. Our algorithm consisted of detection of MC candidates and reduction False-Positives (FPs). To detection MC candidates, top-hat transform in mathematical morphology was used, which is an operation that extracts small elements and details from images. MC candidates extracted from images via top-hat transform contained multiple False Positives (FPs). To reduce FP of MC candidates, we applied Adaboost algorithm in this study. The FP reduction processing included labeling, extraction feature, and Adaboost. First, labeling was performed mask image in MC candidates. Afterwards, the features of MC’s area were extracted using the labeled mc area. Mean and standard deviation of MC’s intensity were employed as feature of MCs. In addition, we used mean, standard deviation differences between surroundings area and MC area. The surrounding area was rectangle area with MC area as a center. Here, the rectangle area had twice width and height against the MC area. We finally performed adaboost using the extracted features. Results The DBT scans were acquired with prototype tomosynthesis system of Korea Electrotechnology Research Institute (KERI) in the breast imaging research laboratory at the Asan Medical Center. The performance of our algorithm measured by k-fold cross-validation is then the average of the values computed in the loop. In our experiment, k was 5. This result showed that our proposed method has sensitivity of 93.2 %. Conclusion In this paper, we have implemented MC detection from SM. We confirmed sensitivity performance of 93.2 % in SMs, which let us confirm the quality of our SMs-generation method. In the future, we will verify the proposed method through more experiments and improve detection method of Micro-Calcification Cluster (MCC) (Fig. 1).

Fig. 1 The performance of MC detection using adaboost Acknowledgments We would like to acknowledge the financial support from the R&D Convergence Program of NST (National Research Council of Science & Technology) of Republic of Korea (Grant CAP-13-3-KERI). References [1] Jeong J, Chae S, Lee S, Chae EY, Kim HH, Choi YW (2015) Simplified false-positive reduction in computer-aided detection scheme of clustered microcalcifications in digital breast tomosynthesis, in proc. SPIE Medical Imaging, vol 9414. [2] Chae S, Jeong J, Lee S, Chae EY, Kim HH, Choi YW (2015) Generation of synthetic mammograms from digital breast tomosynthesis, in proc. cars 2015, S199–S200.

Radiological structured reporting by portable devices M. Fallah1, M. Fatehi1, R. Safdari2 1 Medical Imaging Informatics Research Center, Tehran, Iran, Islamic Republic of 2 Tehran University of Medical Sciences, Tehran, Iran, Islamic Republic of Keywords Structured reporting  Radiology reporting  Reporting templates  M-health Purpose Although structured reporting is considered one of the best choices for modern electronic reporting in radiology practice and there are multiple advantages in approach but not all radiologists are convinced and comfortable to use this method to compose their reports. User interface and data entry issues are among the barriers to discourage reporters to go through structured method. Platforms using portable personal information management devices like smart-phone and tablets may increase willingness of radiologists to apply structured reporting in their practice because of touch-based interactions. The purpose of this paper to explore possibility of using phone or tablet for data-entry and production of structured radiology reports and assessing the user satisfaction.

123

Int J CARS Methods We have developed an Android-based structured reporting system for interpretation of musculoskeletal imaging procedures using RSNA template library (radreport.org). Although the templates are applicable directly to define structured reporting platform but we categorized items included in each template into separate modules each activated through a touch menu. So, the user can select the template, then the specific module or sub-part of the template relevant to the case being interpreted and then details of alteration is defined by just a few clicks. Finally the report can be exported and a PDF file and delivered to the referring physician. In order to evaluate efficiency and user friendliness of the application we asked 5 radiologists to use the application for interpretation of 10 MSK MRI studies and a ‘‘Likert Scale’’ questionnaire was used to collect their feedback in this regard. Results The system has provided a platform to develope touch-based menudriven reporting tol built upon RSNA template library. So, the methods can be used for the rest of the template library not limited to MSK templates. The application was well accepted by the users according to the mean Likert scores The interface design got 7.8 (out of 10). Application information and messages got 7.9 while ease of use got 6.8 score. Overall system features scored 7.15. Conclusion The applications seems to be practically convenient to use. The system can be used by a practicing or under-training radiologist to compose structured reporting based on RSNA library using his/her own portable device from anywhere at any time. The concept may be clinically used for other subspecialties and increase practical application of RSNA templates and structured reporting methods.

Spatiotemporal statistical shape model construction for longitudinal brain deformation analysis using weighted PCA S. Alam1, S. Kobashi1,2, R. Nakano1, M. Morimoto1, S. Aikawa1, A. Shimizu3 1 University of Hyogo, Graduate School of Engineering, Himeji, Japan 2 Osaka University, WPI Immunology Research Center, Osaka, Japan 3 Tokyo University of Agriculture and Technology, Tokyo, Japan Keywords Statistical shape model  Spatiotemporal statistical sha  Weighted PCA  Brain deformation Purpose It is effective to detect neonatal brain diseases and to predict childhood disease onset in neonates because a very-early intervention will decrease the severity of symptoms and significantly improve the quality of life (QOL) of not only children but also their family. It is well known some brain diseases will deform the brain shape. There are some studies to analyze the brain deformation in neonates. For example, they are studying non-rigid brain shape registration with respect to sulcal distribution, brain region segmentation based on fuzzy object model, cerebral contour extraction with sub-voxel accuracy, etc. The remained difficulty of analyzing the neonatal brain shape is a lack of standard brain, which can be used to compare the individual brain shape, and to support brain shape analysis, because the neonatal brain shape is rapidly deforming with growing. This study introduces a spatiotemporal statistical shape model (stSSM), which is an extension of statistical shape model (SSM) in the temporal domain. The stSSM will represent not only the statistical variability of shape but also a temporal change of the statistical variance with growing. The paper proposes a method of constructing stSSM based on weighted principal component analysis (PCA). To

123

validate the proposed method, it has been applied to neonatal brains whose revised age was between -30 days and 46 days, and adult brains whose age was between 43 years and 70 years. Methods 17 neonatal subjects were collected, and 3-D T2-weighted magnetic resonance (MR) images were acquired using 3.0T MRI (Achieva 3.0T TX, Philips Medical Systems, USA) with a spatial resolution of 512 9 512 voxels, and a voxel dimension of 0.75 9 0.75 9 0.75 mm. This study employed 58 adult subjects (age: 40–70 years old) stored in OASIS database. The proposed method applies a weighted-PCA using a temporal weight function. The feature vector of representing brain shape is extracted by means of a level set algorithm, which assigns negative values inside the brain and positive values outside the brain. This study utilizes expectation-maximization (EM) based PCA, in which E-step estimates Eigenvalues of every data using temporal Eigenvectors, and M-step updates Eigenvectors so that it maximizes the variance. E- and M-steps are iterated until convergence of updating vectors. The proposed method assigns a weight parameter for each subject according to subjects’ age, and calculates the weighted variance. Let ti be a time point, and the method constructs the SSM at the time point. The weight function is defined as a Gaussian function, whose center is ti, and variance is a predefined parameter. That is, subjects near ti are dominant to decide the Eigenvectors. By shifting the ti at a short interval from the minimum to the maximum age, the method constructs the stSSM whose Eigenvectors changes with growing or aging. Results The method was applied to the neonatal brain MR image and the adult brain MR image. We first segmented the brain region from both of MR images, and calculated the signed distance map to extract shape features. Figure 1 shows the stSSM of neonatal brain between 10 days and 20 days. The results confirmed that the statistical variety of brain shape changes with growing. The constructed stSSM was evaluated by two measures, generalization ability and specificity. Generalization ability evaluates a performance of representing new acceptable models, and specificity evaluates reconstructing the brain shape via stSSM. The higher values of both measures show the better model representation. The evaluation criteria used was Jaccard Index (J.I.), and evaluation method was leave-one-out cross validation. Mean ± standard deviation of generalization and specificity of adults were 0.89 ± 0.03 and 0.90 ± 0.00, respectively. Those of neonates were 0.66 ± 0.09 and 0.75 ± 0.05, respectively. The performance in neonates was lower than those in adults because of the limited number of subjects.

Fig. 1 Spatiotemporal statistical shape model of neonates Conclusion This paper introduces a spatiotemporal statistical shape model construction method based on weighted PCA. To validate the method, it was applied to neonatal brain growing modeling and adult brain atrophying modeling. The modeling performance was evaluated by generalization ability and specificity. The remained work is to develop a brain shape analysis method using the constructed stSSM.

Int J CARS FFT-based algorithm for feature guided registration of X-ray images for image pasting with masking support S. Ranjan1, B. Patil1, U. Patil2 1 GE Global Research, Bangalore, India 2 Manipal Health Enterprises Pvt. Ltd., Radiology, Bangalore, India Keywords Image pasting  Multistation image acquisition  Feature guided registration  FFT-based computation Purpose The size of the detector in digital X-Ray systems is often not big enough to capture the entire anatomy needed for radiological evaluation in a single exposure, for example, in spine, limb or peripheral run-off angiography studies. The solution is to take multi-station exposures, register them and stitch them together (Fig. 2). Registration of multi-station X-Ray images poses few unique challenges. First, X-ray machines provide very limited positional information between exposures, so registration initialization can at best be very coarse needing a bigger search space. Second, the images may have large dose differences compounded by saturation effects. Third, presence of noise and artefacts need special handling, such as in gradient computation. Fourth, patient motion, both anatomical and physical, is an issue. Lastly, presence of foreign objects whose movement may affect registration, needs masking. In this paper, we propose as approach which address the challenges mentioned and a fast FFT-based algorithm for global search for the optimal registration transform between images.

Fig. 1 Algorithm flowchart

Fig. 2 In the left, three multi-station acquisitions of the spine, and the pasted image. White marks on the pasted image indicate vertical pasting locations computed by registration; three other pasted cases of different scenarios are shown on the right Methods In the absence of a good initialization for registration, and instability of image gradients due to presence of noise, a brute-force search for registration is needed in multi-station X-Ray image registration. The multi-station images in X-Ray are acquired through an automated dose control unit, resulting in very wide dose differences, which makes the pixel values in overlapping region different in different station images. These effects are compounded by image saturation effects. These characteristics of the image necessitate an image normalization step and use of feature-based metric for robust registration. Presence of poissonian noise requires variance normalization cum noise elimination step before image normalization. We use the Ascombe’s transform for variance normalization. Then we do X-Ray acquisition physics guided image normalization for dose compensation between images to be registered. The image domain is transformed to a feature space which is found to be optimum for registration. We then use template matching technique for registration. Algorithm also facilitates incorporation of mask to exclude external influences affecting registration [1] through enhancing the feature distance compute metric. We use information-theory based computation for template selection, and utilize a FFT-based framework for efficient brute-force computation of a feature-based metric of the template over the entire search space [2]. The global maxima of the metric gives the registration transform for stitching. Figure 1 given the flowchart of the proposed algorithm. Results We obtained 115 patients data from GE Definium 8000 Digital X-Ray system, and the number of stations varied from two to seven in these cases. These were angulated acquisitions with minimal parallax effects. Of the 115 data-sets, 43 datasets were chest anterior-posterior acquisitions, 31 were chest lateral acquisitions and 41 were limb acquisitions. A set of reference standards for registration of the subimages for pasting were obtained through manual registration done by an experienced radiologist. We compared our registration results with the reference standard by computing mean square error between the registration transforms. We also defined a performance improvement criteria which computes improvement in results over failure cases using existing state-of-the-art algorithm (see Table 1). The mean, standard deviation, and bounds of error for performance evaluation were analyzed (see Table 2). Results were found acceptable, both quantitatively and qualitatively, by the expert.

123

Int J CARS Table 1 Performance evaluation measure Performance Improvement Measure

ℎ

= 100 – Product Success Rate = 100∗( ℎ )/( = Algorithm Success Rate – Product Success Rate

Table 2 Algorithm evaluation results Protocol

Chest (A-P) Chest (L)

Limb

#patients

43

31

41

#registrations

83

51

66

Error mean and Std Dev 4.2 ± 3.6 (in pixels, 1 pixel * 0.189 mm)

3.6 ± 2.4

4.3 ± 3.9

Error \ 1 mm (% improvement)

81 % (70 %)

75 % (100 %) 73 % (68 %)

Error \ 2 mm

95 % (96 %)

100 % (100 %)

interventions inherit drilling into mastoid cells and trabecula, we recently proposed a registration based on endoscopy inside of these drill holes [1]. A promising application for this registration technique is minimally invasive cochlear implantation [2]–[4]. During this approach, a linear hole is drilled into the temporal bone to the cochlea through a narrow passage, i.e. the facial recess. This procedure demands for high drilling accuracy, since the temporal bone contains risk structures like corda tympani, external ear canal and ossicles. Harming these structures is not tolerable and should be avoided in any case. Figure 1(a) sketches endoscopy of such a drill hole in the temporal bone. A cross-sectional image of the Visible Ear data set [5] shows the target region. Endoscopy of the drill hole records mastoid cells as features and yields image data for registration to preoperative image data. Registration result determines if drilling trajectory needs adjustment or can be proceeded.

94 % (87 %)

Conclusion In this paper, we have provided a framework for effective registration of multi-station X-Ray images. These images have partial overlap, and thus the algorithm may be used for registration of X-Ray images with partial overlap in general. The registration consists of two steps: image normalization, and template search. We have provided method for image normalization for every pair of X-Ray images guided by X-Ray acquisition physics. We do a domain transformation to features found optimal for registration. We then use a FFT-based template matching scheme for obtaining optimal registration transform between sub-images. All sub-images of an acquisition are combined using the registration transforms to produce image containing entire anatomy for radiological evaluation. References [1] Padfield D (2012) Masked Object Registration in the Fourier Domain, IEEE Trans. Image Processing, vol. 21, no. 5, 2706–2718. [2] Reddy BS, Chatterji BN (1996) An FFT-based technique for translation, rotation, and scale-invariant image registration, IEEE Trans. Image Processing, vol. 5, no. 8, pp. 1266–1271.

Image-to-physical registration based on endoscopy of a drill hole inside bone J. Bergmeier1, D. Daentzer2, O. Majdani3, T. Ortmaier1, L. A. Kahrs1 1 Institute of Mechatronic Systems, Leibniz Universita¨t Hannover, Hannover, Germany 2 Department of Orthopaedics, Hannover Medical School, Diakovere Annastift, Hannover, Germany 3 Department of Otolaryngology, Head and Neck Surgery, Hannover Medical School, Hannover, Germany

Fig. 1 a) Endoscopy of temporal bone during minimally invasive cochlear implantation. After pilot drilling, the endoscope is inserted into the drill hole to image the drill hole surface. b) Imaging and unrolling procedure. Top: Endoscope insertion and recording inside a drill hole in combination with the unrolled cylinder surface. Red rectangles mark the area, imaged by the endoscope. Bottom: Unrolling of drill hole surface into a plane image

Keywords Trabecula  Mastoid  Spongious bone

Several groups have shown that Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) are well suited for volumetric imaging of mastoid cells [3] and spongiosa. Such data sets will be used as the image part of our image-to-physical registration. Main advantage of our new registration method is the high number and density of clearly visible landmarks inside both modalities: image

Purpose Mastoid cells as well as trabecula provide unique bone structures that can be applied as natural landmarks for registration. Preoperative imaging is able to sufficiently image these structures, but registration requires an intraoperative counterpart. Since versatile surgical

123

Int J CARS (volumetric data set) and physical (intraoperative endoscopy). Intraoperative CT or MRI imaging, artificial landmarks, etc. might become obsolete. Methods Recording of the drill hole surface yields bone-air patterns that are applied as intraoperative registration feature. In this contribution, we discuss an approach that unrolls the drill hole surface into a two-dimensional image. Core of our registration approach is comparison of the drill hole surface, imaged on the one hand with an endoscope and on the other hand simulated from an existing image data set. A series of simulated surfaces is generated, resulting each from a different drill hole pose. Unrolling of these surfaces into flat 2-D images allows for intensity-based 2-D image registration. Figure 1(b) shows the endoscopic imaging and unrolling procedure of a drill hole cylinder surface. On top, a half cylinder surface is shown in combination with an angled view endoscope, inserted into the drill hole. The red rectangle marks the area of the drill hole, imaged by the endoscope. Next to the half cylinder, the unrolled drill hole surface is shown as flat image that is also overlaid with a red rectangle, which shows the same area, imaged by the endoscope. Below, the extracted surface and the unrolling into a plane image are shown. In the first registration step, the endoscopic image is rigidly registered to each simulated image by means of a 2-D/2-D registration, using conventional intensity based registration methods, e.g. Mutual Information. An optimization algorithm aligns the endoscopic image as good as possible onto the image of the data set, yielding a value for this best alignment as result of one 2-D/2-D registration. Repeating this process for all images in the search space results in a number of registration metric values, each representing the similarity of the endoscopic image to one image from the volumetric data set. Finally, a global registration step is performed to evaluate the best registration result of the subsequently performed local registrations. This best result corresponds to a certain simulated drill hole, providing a drill hole pose in the preoperative image data set. Simulation of workflow allows for investigation of the registration procedure without caring about image data acquisition and processing. Both, preoperative and endoscopic image data are created artificially, using the Visible Ear data set [5]. It consists of image slices, showing a quarter-head, with a voxel size of 0.05 mm3 for the employed slices 70 to 355. This provides high resolution and enables simulation of image data with lower resolution by artificial deterioration. Results Experiments are performed with deteriorated image data using voxel size of 0.2 mm3 and multiplicative Gaussian noise of variance 5. A search space with radius 2 mm is sampled with 10201 drill trajectories. Registration results are shown in Fig. 2 as scatter plot. Each scatter point shows the result of the best registration of the endoscopic image to one image from a drill hole (2-D/2-D registration) in the search space. Height indicates similarity of both images, while lateral position marks where the center of the drill trajectory is located in the search space. Ground truth is given by registration of the endoscopic image to the unrolled image of the known correct drill hole (center of search space center in Fig. 2). Distribution of the results shows distinct characteristics and maximum values of an envelope function.

Fig. 2 Scatter plot, showing results after 2-D/2-D registration. Height and color of the plot show registration results. Results are plotted according to the positions of the trajectory centers Conclusion This contribution demonstrates basic methods for registration with endoscopic acquisition of small sized features like trabecula or mastoid cells for image-guided procedures and has the potential to revolutionize bone registration. We presented methods that make use of specific bone-air patterns, like trabecular bones and mastoid cells, as registration features for image-to-physical registration. Endoscopy of these patterns at the drill hole surface provides image information that allows for registration to preoperative image data of the drill pose. The proposed method unrolls the drill hole surfaces into flat 2-D images and thus enables utilization of conventional 2-D registration methods like Mutual Information. As a first proof-of-concept, the procedure has been tested using simulated image data. Image data basis are high-resolution crosssectional pictures of a quarter head, including mastoid cells as registration features. Simulation of the procedure has shown general functionality of the proposed method. Current work deals with processing of real endoscopic image data for registration with simulated images from preoperative image data. Therefore endoscopic recordings are dewarped and stitched to achieve the desired flat 2-D images of the drill hole surface. First intraoperative examinations revealed challenges due to body fluids like blood or bone marrow. Suction or irrigation might then improve imaging. Also, the influence of drilling onto the bone structure will be investigated. References [1] Bergmeier J, Daentzer D, Noll C, Majdani O, Ortmaier T, Kahrs LA (2015) Towards endoscopic image-to-physical registration of mastoid cells and trabecula. Annual Conference of the German Society for Computer and Robot assisted Surgery (CURAC), 14, 43–48. [2] Bell B, Williamson T, Gerber N, Gavaghan K, Wimmer W, Kompis M, Weber S, Caversaccio M (2014) An image-guided robot system for direct cochlear access. Cochlear Implants Int., 15(1), 11–3.

123

Int J CARS [3]

[4]

[5]

Labadie RF, Balachandran R, Noble JH, Blachon GS, Mitchell JE, Reda FA, Dawant BM, Fitzpatrick JM (2014) Minimally invasive image-guided cochlear implantation surgery: First report of clinical implementation. Laryngoscope, 124(8), 1915–1922. Majdani O, Rau TS, Baron S, Eilers H, Baier C, Heimann B, Ortmaier T, Bartling S, Lenarz T, Leinung M (2009) A robotguided minimally invasive approach for cochlear implant surgery: Preliminary results of a temporal bone study. Int. J. Comput. Assist. Radiol. Surg., 4(5), 475–486. Sørensen MS, Dobrzeniecki AB, Larsen P, Frisch T, Sporring J, Darvann TA (2002) The visible ear: A digital image library of the temporal bone. Orl, 64(6), 378–381.

images in a hierarchical fashion from largest deformations to small and local ones. Once the desired alignment is achieved the transformation and defining target points set, which can range from 8 to 729 points, can be exported to be used as a gold standard to validate the accuracy of automatic registration procedures.

The validation grid: a new tool to validate multimodal image registration I. J. Gerard1, M. Kersten-Oertel1, A. Kochanowska1, J. A. Hall1, D. L. Collins1 1 McConnell Brain Imaging Centre Montreal Neurological Institute, Montreal, Canada Keywords Multimodal image registration  Validation  Image-guided surgery  Brain shift Purpose In many medical imaging applications, there is a need to register images of different modalities in order to facilitate either diagnosis of pathology or planning of a surgical intervention. Multimodal image registration is a non-trivial problem since the information available in each modality may not be clearly related and can require complex techniques in order to correctly align the images. With the development of these techniques, it is important to validate their accuracy in order to quantify the reliability of individual methods. Validation is a difficult task as it traditionally refers to the comparison of a result to the ground truth which is often unavailable. Due to this, many registration techniques are usually validated by comparing their results to the results of other widely accepted registration techniques. We present a new tool for validating multimodal image registration techniques. The tool allows the user to manually align two images by manipulating a series of target points that are placed on a regular shaped grid on both the target and unregistered image. As the points are moved the registration transform is updated based on the displacement of corresponding grid points using a thin plate spline model. The tool has been incorporated within our image-guided neurosurgery (IGNS) navigation platform, IBIS [1] and used to validate intraoperative ultrasound (iUS)—MRI registration for brain shift compensation. Methods The proposed tool was developed within our custom neuronavigation system, IBIS. A full description of the system and the research development features can be found in Mercier et al. 2011 [1]. The Validation Grid, is initialized by placing a regular grid of target points on two different image volumes within the domain of the fixed image (Fig. 1A). The target points on the fixed image are fixed, and exist for the sole purpose of defining the transformation, and the points on the registration target are movable. The set of moving points can be rigidly manipulated as a grid (translation and rotation) or each point can be individually moved to account for more complex non-rigid deformations. The images are registered using a thin plate spline transform defined by the target points and the registration is automatically updated whenever a point is moved. Due to the difficult nature of manually aligning images of different modalities the size of the validation grid can be changed from a coarse 2 9 2 9 2 grid to a finer 9 9 9 9 9 grid allowing the user to manually register the

123

Fig. 1 Initialization of the Validation Grid on an US (orange) and MR (grey) volume. The yellow dots correspond to the points to be manipulated for manual registration Results We present an initial demonstration of the tool on a dataset obtained at the Montreal Neurological Institute and Hospital. The dataset involves a preoperative T1w, gadolinium enhanced MRI image and an intraoperative ultrasound (iUS) volume reconstructed from a series of 2D ultrasound images originally presented in [2]. The manual registration from the validation grid was compared to the result from the gradient orientation alignment algorithm [3] and the single point validation in [2]. The proposed method has a comparable level of accuracy with the additional benefit that the Validation Grid uses a larger set of points and allows for a better description of the overall registration accuracy. The results are summarized in Fig. 1B and the pre- and post- registration is shown in Fig. 1C and D respectively (Table 1). Table 1 Summary of registration results Method

Number of val- Pre-iUS registra- Post-iUS registraidation points tion error (mm) tion error (mm)

Landmark matching [2]

1

5.46

1.36

Validation Grid

125

6.32 ± 1.88

1.63 ± 1.46

Conclusion The presented registration validation tool has the potential to greatly facilitate a more comprehensive way to validate new registration techniques. With the current structure of the Validation Grid, we have removed the traditional need to pick corresponding landmarks on fixed and moving images—a technique that is prone to large localization errors—to validate registration procedures. Of course the usefulness of the proposed validation tool is directly dependent on the

Int J CARS quality of the manual registration which can be very time consuming. However, with validation sets of up to 700 points, these errors are likely to have a smaller effect on a global evaluation of the accuracy of these procedures compared to sets involving 10–20 points, which is provided in recent validation sets [4]. With further development and validation of the Validation Grid tool through intra- and inter-rater variability assessments this has the potential to become a powerful tool for validating registration algorithms (Fig. 2).

Fig. 2 iUS acquisition overlaid on MRI before and after registration. Orange arrow highlights area where misalignment and its improvement can be visualized References [1] Mercier L, Del Maestro R F, Petrecca K, Kochanowska A, Drouin S, Yan C X, Janke A L, Chen S J and Collins D L. (2011) ‘‘New prototype neuronavigation system based on preoperative imaging and intraoperative freehand ultrasound:system description and validation,’’ International journal of computer assisted radiology and surgery, 6(4):507–522. [2] Gerard I J, Kersten-Oertel M, Drouin S, Hall J A, Petrecca K, De Nigris D, Arbel T and Collins D L. (2016) ‘‘Improving Patient Specific Neurosurgical Models with Intraoperative Ultrasound and Augmented Reality Visualizations in a Neuronavigation Environment,’’ in 4th Workshop on Clinical Image-based Procedures: Translational Research in Medical Imaging, LNCS 9401:28–39. DOI: 10.1007/978-3-319-31808-0_4 [3] De Nigris D, Collins D L, and Arbel T. (2012) ‘‘Fast and robust registration based on gradient orientations: case study matching intra-operative ultrasound to pre-operative mri in neurosurgery,’’ in Information Processing in Computer-Assisted Interventions, pp 125–134, Heidelberg. [4] Mercier L, Del Maestro R, Petrecca K, Araujo D, Haegelen C, and Collins D L. (2012) ‘‘Online database of clinical MR and ultrasound images of brain tumours,’’ Medical Physics, 39(6):3252–3261.

Asymmetric and multi-layered object for geometry calibration between X-ray source and detector for portable X-ray imaging system K. Sato1, T. Ohnishi2, M. Sekine2, H. Haneishi2 1 Chiba University, Engineering, Chiba, Japan 2 Chiba University, Center for Frontier Medical Engineering, Chiba, Japan Keywords Portable X-ray imaging system  Geometry calibration  Calibration object  Registration Purpose Portable X-ray imaging system is widely used for round and emergency medicine. It can reduce its weight by using flat panel detector (FPD) and allow the free placement of X-ray source and

FPD. If the geometry between X-ray image and detector at image acquisition is obtained, the utility of the system is further enhanced. For example, three-dimensional image such as computed tomography and tomosynthesis may become possible using X-ray images and the geometry information. We have proposed an image processing-based calibration method using a flat calibration object (CO) attached to the X-ray source housing [1]. In such method, large error took place in out-of-plane direction because the CO did not have depth information. In this paper, we propose an asymmetric and multi-layered calibration object (AMCO) for accuracy improvement in calibration. Methods Figure 1 shows a schematic illustration of portable X-ray imaging system with an AMCO and its calibration method. The AMCO is always attached to the X-ray source during target image acquisition. Its length of one side is 100 mm and thickness is 10 mm. The dimensions of the AMCO is known and the positional relationship between X-ray source and the attached AMCO can be obtained in advance by another calibration technique [1]. The geometry between the X-ray source and the detector is estimated through 2D-3D image registration of the AMCO. Namely, a digitally reconstructed radiograph (DRR) is obtained through ray projection [2] of a virtual AMCO constructed in a computer. Then, geometry parameters are determined by maximizing the normalized cross correlation [3, 4] between the DRR and the real X-ray image of the AMCO.

Fig. 1 Proposed calibration method using AMCO If CO is a single layer, the registration accuracy for out-of-plane direction becomes significantly lower than in-plane one. We avoid this problem with the double-layered CO. This depth difference between layers is reflected on the projection image as different magnification. Furthermore, the shape and the placement of the CO patterns are designed asymmetrically in order to avoid the local solution in optimization. Results We conducted an experiment to evaluate the calibration accuracy. The detector was placed on a XY positioning stage whose position can be translated in in-plane with 0.01 mm pitch. Then X-ray source was fixed and X-ray source-detector distance (SDD) was kept around 900 mm. 34 X-ray images of the blank object space were acquired as moving the positioning stage in only in-plane direction because the stage cannot be translated in out-of-plane direction. To confirm the effect of AMCO, we compared the one CO case with two COs case in the following way. At the initial setup, the

123

Int J CARS vector from the X-ray source to the detector center (S-D vector) was accurately estimated by additional calibration method using metallic markers [1]. Here, the S-D vector obtained at this setup was defined as initial S-D vector. For each position of the stage, the S-D vector was estimated by proposed calibration method. Then the difference between the actual S-D vector and the estimated S-D vector was evaluated by root mean square error (RMSE). The actual S-D vector can be calculated from initial S-D vector and a known moving distance of the stage. As for out-ofplane direction, variability in z-component of the estimated S-D vector was evaluated. Table 1 shows the result. RMSEs for in-plane direction and outof-plane direction by one CO were 1.1 mm and 6.5 mm, whereas results by two COs were 1.7 mm and 1.2 mm. We confirmed that RMSEs for both directions were less than 0.2 % relative to SDD. In this table, the result of our previous study with a thin and flat CO is also shown. In such experiment, the in-plane error was 3.8 mm and out-of-plane error was 9.1 mm. From these results, we found that the in-plane error was reduced by asymmetry patterns and the out-ofplane error was also reduced by stacking two COs. Out-of-plane error using one thick CO is even better than the previous result. It suggests that even one thick CO is effective in improving the accuracy. The accuracy could be further improved by optimizing the material and pattern of the AMCO. Table 1 Calibration error Thickness of object

Number of objects

Patttern placement

In-plane direction [mm]

Out-of-plane direction [mm]

Thin (previous)

1

Symmetry

3.8

9.1

Thick (10 mm)

1

Asymmetry 1.1

6.5

2

Asymmetry 1.7

1.2

Conclusion In this paper, we designed the calibration objects to improve the registration accuracy in out-of-plane direction. The AMCO is composed of two asymmetrical-shaped thick plates. From experiment results, we found that calibration accuracy with the AMCO was higher than that of previous CO in both in-plane direction and out-ofplane direction. As a future work, we will study the feasibility of the proposed calibration method in practical application such as tomosynthesis. References [1] Ohnishi T, Suganuma S, Takano Y, Haneishi H (2014) Freehand digital tomosynthesis using portable X-ray imaging system with real-time calibration. CARS 9(1). [2] Lactoute P, Levoy M (1994) Fast Volume Rendering Using a Shear-Warp Factorization of the Viewing Transformation. Proc. SIGGRAPH’94: 451–458. [3] Holden M, Hill D L.G., Denton E R.E., Jarosz JM, Cox T C.S., Rohlfing T, Goodey J, Hawkes DJ (2000) Voxel Similarity Measure for 3-D Serial MR Brain Image Registration. IEEE Trans Med Imaging 19(2): 94–102 [4] Skerl D, Likar B, Pernus F (2006) A Protocol for Evaluation of Similarity Measures for Rigid Registration. IEEE Trans Med Imaging 25(6): 779–791.

Segmentation of hyper-acute ischemic stroke based on random forest from diffusion weighted imaging Y. Peng1, X. Zhang1, P. Gao2, J. Xue2, W. Li3, L. Ren3, Q. Hu1 1 Research Lab for Medical Imaging and Digital Surgery, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China 2 Beijing Tiantan Hospital, Capital Medical University, Beijing, China 3 Shenzhen Second People’s Hospital, Shenzhen, China Keywords Hyper-acute ischemic stroke  Segmentation  Feature selection  Random forest Purpose Hyper-acute ischemic stroke is an important brain injury because of decrease in or deprivation of oxygen and blood supply. It is hard to delineate ischemic stroke regions due to diverse locations, variable shapes and unclear boundaries even for experienced experts. At hyper-acute stage, diffusion weighted (DW) imaging is preferred due to its high sensitivity in detecting stroke lesions [1]. Therefore an automated and effective method is necessary to segment ischemic stroke to aid the decision making for treatment. We investigate random forest on hyper-acute ischemic stroke segmentation from baseline DW imaging (including T2-weighted image, DW image (DWI) and apparent diffusion coefficient (ADC)). Methods Altogether 84 patients were recruited in the study, within 6 h from onset, with baseline DW imaging. The ground-truths of hyper-acute ischemic stroke on DWIs were manually drawn from an experienced radiologist. First, we calculate asymmetry map (ASM) to augment the features to classify [2]. To alleviate the interference of normal brain tissues and speed up the calculation, candidate infarcts in the form of regions of interest (ROIs) are extracted from thresholding ADC maps subjected to DWI [3]. Twenty-five features are extracted to represent various features of hyper-acute ischemic stroke from each map. We included intensity, statistical features (mean, minimum, maximum, median, Gaussian average, standard deviation, kurtosis, skewness) within and neighbors, and histogram within windows. The total number of features is 100. ReliefF algorithm is employed to remove feature redundancy [4]. Then, random forest is applied to learn the segmentation model of two classes, with its parameters optimized through two-fold cross validation. In training phase, positive training samples are all voxels of the expert drawn lesion, while negative training samples consist of voxels that are included through morphological dilation around the positive samples. In testing phase, we input ROIs to classification model to achieve the segmented hyperacute ischemic stroke. The diagram of the proposed method is shown in Fig. 1.

Fig. 1 Flowchart of the proposed method

123

Int J CARS Results We construct random forest classifier using the selected features. The proposed method is implemented on different numbers of features selected from ReliefF algorithm. Dice similarity index (DSC) is computed to evaluate the segmentation accuracy. The results are shown in Fig. 2. It is found that the highest DSC is with all features. Different numbers of features are tested on the classifier. It is noted that when the number of selected features is 75, the change in DSC is very minor.

Fig. 2 The DSC obtained using different numbers of features Conclusion We have proposed a method to segment hyper-acute ischemic stroke from DW imaging by constructing the classification model of lesion. This technique contributes to more accurate segmentation of ischemic stroke, which is considered challenging due to unclear boundaries between ischemia and its neighboring brain tissues. The ASM map contributes greatly to the classification of ischemic stroke, which might be extended for other image classification even if there is grayscale overlap between different image classes. It can be noted that a good DSC can be attained with 75 features. The proposed method could be a potential tool for clinicians to quantify the hyperacute ischemic stroke and assist the decision making especially for thrombolytic therapy. Acknowledgements This work has been supported by: National Program on Key Basic Research Project (Nos. 2013CB733800 and 2013CB733803), National Science and Technology Pillar Program during the Twelfth Five-year Plan Period (No. 2011BAI08B09), Shenzhen Key Technical Development Grant (No. CXZZ20140610151856719), and Shenzhen Basic Research Grant (No. JCYJ20140414170821262). Authors would like to thank Dr. Yiqun Zhang for her valuable discussion on manually delineating infarct regions without clear boundaries. References [1] Muir KW, Buchan A, von Kummer R, Rother J, Baron JC (2006) Imaging of acute stroke. Lancet Neurol 5(9): 755–768. [2] Peng YQ, Zhang XD, Hu QM (2015) Segmentation of hyperacute ischemic infarcts from diffusion weighted imaging based on support vector machine. J. Comput. Commun. 3(11): 152. [3] Ma L, Gao PY, Hu QM, Lin Y, Jing LN, Xue J (2010) Liu ML. Prediction of infarct core and salvageable ischemic tissue volumes by analyzing apparent diffusion coefficient without intravenous contrast material. Acad. Radiol 17(12): 1506–1517. [4] Robnik-Sikonja M, Kononenko I (2003) Theoretical and Empirical Analysis of ReliefF and RReliefF. Mach. Learn. 53, 23–69. Mitigating medialness responses from non-tubular structures using entropy M. Unberath1,2, E. Goppert1, O. Taubmann1,2, A. Maier1,2

1

Pattern Recognition Lab, Friedrich-Alexander University ErlangenNuremberg, Germany 2 Erlangen Graduate School in Advanced Optical Technologies (SAOT), Germany Keywords Coronary arteries  Minimal cost path  Shannon entropy  Centerline extraction Purpose Vessel segmentation and centerline extraction is relevant in clinical practice [1, 2]. Recent methods rely on minimum cost path search in a grid storing the edge cost values. Cost values are usually calculated using vesselness or medialness filters [1]. The medialness filter is contrast independent and does not require thorough parameter tuning. However, it yields high responses also at non-circular edges of bright structures, such as bones. We propose to extend the contrast independent medialness filter introduced in [1] with a weighting factor based on the normalized Shannon entropy in order to suppress high responses from non-circular edges. Methods The medialness measure is a gradient-based, multi-scale method targeted at the detection of circular structures. Put concisely, the medialness response mðx0 Þ is highest if x0 is the center of a circle in any cross-sectional plane through x0 that is bright with respect to the background. The medialness measure is given by ( ) N1 1X ~ 0 ; RÞg ¼ max Eðyi ðRÞÞ ; ð1Þ mðx0 Þ ¼ maxfmðx R R N i¼0

where yi ðRÞ ¼ x0 þ Ruðai Þ are samples at N angles ai , Eðyi ðRÞÞ / maxfrr I ðyi ðRÞÞg is the normalized, contrast independent edge r response of image I ðxÞ at scale r [1], and R is the radius of the structure. The line uðai Þ ¼ sinðai Þ  u1 þ cosðai Þ  u2 lies on a crosssectional image plane through x0 that is spanned by the basis vectors u1 and u2 . The medialness measure yields high responses not only for circular structures but also for other objects with strong negative gradients. In order to suppress high responses from non-circular structures while maintaining contrast independence, we propose to use the normalized Shannon entropy as a weighting factor. In case of a circular structure, we can assume that the edge responses of sample points Eðyi ðRÞÞ on a circle with radius R have approximately the same magnitude for all angular samples ai , as we expect similar gradients at its boundaries. Then, the magnitude of the edge responses over all angular samples can be interpreted as a distribution, allowing the computation of its entropy. The entropy is related to the randomness of a distribution and is highest in case of uniform distributions [3]. To preserve normalization of the medialness, we use the normalized Shannon entropy H ðx0 ; RÞ ¼ PN1 Eðyi ðRÞÞlnðEðyi ðRÞÞÞ of all samples Eðyi ðRÞÞ with  lnð1N Þ i¼0 PN1 radius R, fulfilling i¼0 Eðyi ðRÞÞ ¼ 1. This leads to entropies close to 1 if all angular samples yield edge responses with similar magnitude and values close to zero otherwise. For the purpose of increasing the effect of punishing non-circular structures, the quadratic entropy ~ 0 ; RÞ yielding the is multiplied to the radius-dependent medialness mðx entropy-supported medialness   ~ 0 ; RÞ  H 2 ðx0 ; RÞ : ~ðx0 Þ ¼ max mðx ð2Þ m R

We expect this extension to leave the response in the center of a circle largely unchanged but to mitigate the response away from the center and at non-circular boundaries. In order to assess the performance of the novel measure quantitatively, we conduct a phantom study using a volumetric B-spline phantom based on the XCAT [4] showing the thorax with contrasted

123

Int J CARS coronary arteries at diastole. Then response volumes are calculated for both the native medialness according to [1] and the entropysupported medialness proposed here. Finally, the average response for both cases is calculated along the arteries’ centerlines, within the arteries, and within the whole volume. Moreover, both medialness filters are applied to a four-chamber enhanced CT angiography scan of a 45 year old female. Results The results of the phantom study are summarized in Table 1. As expected, the average response along the centerlines and inside the arteries is only marginally affected. The mean entropy-supported medialness response of the whole volume, however, is two times smaller than for the native medialness filter. The results indicate that responses from non-tubular structures are suppressed effectively in the proposed approach. This observation is confirmed qualitatively when considering volume renderings of the response volume of the real CT angiography data set that are shown in Fig. 1. Table 1 Quantitative results of the phantom study Along centerline

Within arteries

Whole volume

Native

0.95 ± 0.057

0.79 ± 0.12

0.040 ± 0.082

Entropysupported

0.95 ± 0.059

0.78 ± 0.14

0.020 ± 0.049

The results are stated as the average response together with the corresponding standard deviation

Conclusion The entropy-supported medialness successfully suppresses responses for structures without circular cross section such as edges while leaving responses of tubular structures such as vessels largely unaffected. The preliminary results presented here encourage a more thorough evaluation on publically available data sets [2]. We believe that the entropy-supported medialness may outperform the native medialness in scenarios where streaking artifacts degrade the image quality [5]. References [1] Gu¨lsu¨n M A et al. (2008) Robust Vessel Tree Modeling. MICCAI 2008, pp. 602–611. [2] Metz C et al. (2008) 3D Segmentation in the Clinic: A Grand Challenge II—Coronary Artery Tracking. Insight Journal 1(5):1–12. [3] Shannon C E (2001) A mathematical theory of communication. Mobile Computing and Communications Review 5(1):3–55. [4] Segars W P et al. (2010) 4D XCAT phantom for multimodality imaging research. Med Phys 37(9):4902–4915. [5] Schwemmer C et al. (2013) Residual motion compensation in ECG-gated cardiac vasculature reconstruction. Phys Med Biol 58(11):3717

Knowledge-based automatic segmentation of the globes and optic nerve on CT images N. Aghdasi1, Y. Li1, A. Berens2, R. Harbison2, K. Moe2, B. Hannaford1 1 University of Washington, Electrical engineering, Seattle, United States 2 University of Washington, Head and Neck Surgery, Seattle, United States Keywords Optic nerve  Eye globe  Automatic segmentation  Skull base surgery

Fig. 1 Volume renderings of response volumes obtained with the native medialness filter (left) and the entropy-supported medialness (right) using the same transfer function

123

Purpose In preoperative planning stage for skull base surgery, the optic nerve and globes are critical regions that need to be outlined and avoided during the procedure. Manual delineation using drawing tools by a trained expert is the most commonly used segmentation technique, which is time-consuming and suffers from large inter- and intra-rater variability. The use of automated image segmentation saves time, effort, and increases precision. However, the small size of these organs and the similar density of the surrounding tissues make segmentation difficult. Considering the complex geometry, conservation of anatomic morphology, symmetry, and spatial relationships of structures in head and neck CT scans, employing prior anatomical knowledge is critical for accurate segmentation [1]. We are interested in the development of an efficient technique for the segmentation of the optic nerve and globes which uses anatomical landmarks and prior anatomical knowledge to perform the segmentation task. Particularly, the segmentation is done by combining the anatomical knowledge from clinicians with image processing techniques. Utilizing prior knowledge and reliable anatomical landmarks to define an optimal Region of Interest (ROI) which contains the desired structures is an effective way for fast localization and successful segmentation. This approach is efficient and robust to CT data with variable voxel resolution and does not require training data sets. Methods The proposed method progressively locates anatomical landmarks to restrict the original volume to a smaller volume, which has a higher

Int J CARS probability of containing the desired structure (i.e. optic nerve and globe). The skull base area was defined as initial ROI and was determined by detecting the nasal tip point and anterior aspect of the frontal bone in tissue and bone thresholded volume respectively. The initial skull base ROI, was restricted more by detecting additional anatomical landmarks. The optic nerve and globe are located inside the orbit (eye socket), and the bony orbit protects these delicate structures [2]. Therefore, the second ROI was defined as a volume inside the orbit and was determined by finding the inner boundaries of the orbit from analyzing 2D slices (left and right orbit separately). Hence, the final ROI contains the globe, optic nerve and extraocular muscles, Fig. 1.

Fig. 1 Skull base ROI—left to right: 1) Original bone volume. 2) New volume by cropping the original volume using center of mass. 3) Nasal point position in tissue thresholded volume. 4) Nasal point position and frontal bone position. 5) Skull base ROI The globe (right or left) was obtained by analyzing the intensity change along vertical and horizontal lines in 2D slices. Globe volume was deduced from the final ROI which leaves out optic nerve and extraocular muscles. The optic nerve and extraocular muscles have similar intensity, however surrounding tissue of the optic nerve, which separates it from the extraocular muscles has lower intensity. The Otsu’s method was used to find the threshold intensity to segment the foreground (optic nerve and muscles) from background (tissue surrounding optic nerve). Afterwards, the 3D point cloud of both volumes were constructed, Fig. 2. A convex hull was created using the point cloud of tissue surrounding the optic nerve, and entire foreground point cloud inside the convex hull was labeled as optic nerve.

volumetric overlap (Dice metric) was used as criteria in axial slice for quantitative comparison. The average dice coefficient for eye globes and optic nerve (right side) were 0.85 and 0.52 respectively. As recommended by Zijdenbos et al. [3] in the literature of image validation a good overlap occurs when dice coefficient is greater than 0.7. In case of optic nerve, comparing the ground truth to segmentation result revealed our method labeled extraocular muscles as optic nerve. In future work, we are planning to improve the segmentation by defining constraints on optic nerve shape and subtracting the extraocular muscles from the optic nerve segmentation. Conclusion This paper presents an automatic globe and optic nerve segmentation method, using prior anatomical knowledge. The segmentation process is efficient and does not require training data. Detection of reliable anatomical landmarks to restrict the original volume to a smaller volume, reduces the computational cost and sensitivity to image artifacts and improve the precision. References [1] Kobashi M, Shapiro LG (1995) Knowledge-based organ identification from CT images. Pattern Recognition 28.4: 475–491. [2] Milap P, Mehta J, Perry D (2015) Medial Orbital Wall Landmarks in Three Different North American Populations. Orbit. 34,72–78. [3] Zijdenbos AP, Dawant BM, Margolin RA, Palmer AC (1994) Morphometric analysis of white matter lesions in MR images: method and validation. IEEE Trans Med Imaging;13:716–724.

Semi-automatic myocardial segmentation the reproducibility of bright-blood myocardial T2* quantification in multi gradient echo MRI

improves

P. Triadyaksa1,2, N. H. J. Prakken1,3, M. Oudkerk1, P. E. Sijens1,3 1 University of Groningen, UMCG, Center for Medical Imaging-North East Netherlands, Groningen, Netherlands 2 Diponegoro University, Department of Physics, Semarang, Indonesia 3 University of Groningen, UMCG, Department of Radiology, Groningen, Netherlands Keywords Gradient vector flow snake  k-Means clustering  Semiautomatic segmentation  Bright-blood myocardial T2*

Fig. 2 Point cloud of optic nerve and extraocular muscles with convex hull of tissue surronding the optic nerve Results Twelve data sets from Radiation Therapy Oncology Group were used to test the algorithm. The optic nerve manual segmentation (ground truth) was provided and the globes were manually segmented otolaryngologists in our group. The reconstruction matrix for all datasets was 512 9 512 pixels with the pixel size of (0.98 * 1.16) 9 (0.98 * 1.16), and the slice thickness ranged from 2 mm to 3 mm. The evaluation compares results of proposed techniques with the ground truth of expert manual segmentation. The

Purpose Myocardial T2* imaging in bright blood mode is used in clinical practice to non-invasively assess and monitor myocardial iron deposition using a 1.5 T multi gradient echo magnetic resonance sequence (MGE). For post-processing there are two types of image data which may be selected to draw the left ventricular (LV) epicardial and endocardial contours; a single short-axis image, which corresponds to a single acquired echo time (TE), and a composite image, which comprises of a combination of images from the MGE series. In the clinical assessment of myocardial iron deposition, manual myocardial drawing is time consuming and subject to intraobserver and interobserver variability. Even though an automatic method has been presented for segmenting the interventricular septum region using black-blood MGE series, to our knowledge, segmentation remains a challenge, especially in bright-blood MGE series due to the poor contrast between the myocardium and its surroundings on original MGE images. A previous study showed that contrast enhancement by generating a composite image from MGE image series, improved reproducibility between observers in the quantification of myocardial T2* for the myocardium as a whole [1]. The composite image was generated by combining TE images of optimum contrast-to-noise ratio between myocardium and its surroundings (left ventricle blood pool, right ventricle blood pool, and lung). In this

123

Int J CARS study we validated a semi-automatic segmentation method for myocardial T2* quantification using the composite images of 33 short-axis LV slices (7 apical, 18 mid-ventricular, and 8 basal) acquired in a group of haematology and suspected cardiomyopathy patients. Methods On the shortest TE image, a region of interest (ROI) of myocardium was drawn manually (Fig. 1a) and used to create a dark blood pool ROI by subtracting the signal intensity on the first TE from those with optimum contrast-to-noise ratio between myocardium and the LV blood pool (LVBP). K-means clustering [2] was applied on this ROI (Fig. 1b) with removal of the higher cluster values at the periphery leaving the remaining pixels inside the LVBP as a mask for the gradient vector flow snake contouring [3] (Fig. 1c) to determine the initial endocardial contour (Fig. 1d). From the endocardial contour, radial segments consisting of several layers heading outward were created (Fig. 1e). Coefficients of variance (CoV) in pixel signal intensity on the composite image was assessed for all radial segments at each layer expansion of one pixel thickness and stopped when the expanded layer had a CoV exceeding 20 % to effectively locate the border of lateral myocardium (Fig. 1f). For anterior, septal and inferior positions, k-means segmentation was used to differentiate the myocardium from its surroundings (Fig. 1g). By comparing the number of layers to that of each neighbor per radial segment, further correction was added to eliminate pixels at the outer layers per radial segment using the rule; two pixels wider than its neighbor, which resulted in a myocardial contour mask (Fig. 1h). Based on this mask, an optimal myocardial contour detection was generated (Fig. 1i) by adding two pixel layers inward and outward of the myocardial contours creating several combinations of myocardial thickness expansion and compression to find an optimal combination by removing unwanted pixels, which resulted in contours equivalent to manual drawing (Fig. 1j).

model shows lower variability within and between observers by using semi-automatic segmentation as compared to manual drawing (CoV of 11.61 vs. 14.15 and 15.21 vs. 16.21). On short-axis slices with a minimum segment T2* below 20 ms, which represents iron loading pathology, semi-automatic segmentation also improved the intraobserver and interobserver reproducibility with higher DSC (0.86 ± 0.05 vs. 0.79 ± 0.04, P \ 0.001 and 0.88 ± 0.04 vs. 0.74 ± 0.06, P \ 0.001) and lower CoV (15.19 vs. 17.79 and 15.68 vs. 19.34). The same trend was seen for short-axis slices in the absence of iron loading pathology with a DSC of 0.86 ± 0.05 vs. 0.80 ± 0.04, P \ 0.001 and 0.86 ± 0.04 vs. 0.77 ± 0.04, P \ 0.001 within and between observers, and lower CoV (10.46 vs. 12.81 and 14.84 vs. 15.12). Conclusion Semi-automatic myocardial segmentation as assessed on contrastoptimized composite images can be applied in clinical practice for iron loading assessment, This method shows better T2* quantification reproducibility using more consistent and less user dependent myocardial ROIs. References [1] Triadyaksa P, Handayani A, Dijkstra H, Aryanto KYE, Pelgrim GJ, Xie X, Willems TP, Prakken NHJ, Oudkerk M, Sijens PE (2016) Contrast-optimized composite image derived from multigradient echo cardiac magnetic resonance imaging improves reproducibility of myocardial contours and T2* measurement. Magn Reson Mater Phys 29(1):17–27. [2] Jain AK (2010) Data clustering: 50 years beyond K-means. Pattern Recognition Letters 31(8):651–6. [3] Xu C, Prince JL (1998) Snake, shapes and gradient vector flow. IEEE Trans on Img Processing 7(3):359–369. [4] Positano V, Pepe A, Santarelli MF, Scattini B, De Marchi D, Ramazzotti A, Forni G, Borgna-Pignatti C, Lai ME, Midiri M, Maggio A, Lombardi M, Landini L (2007) Standardized T2* map of normal human heart in vivo to correct T2* segmental artefacts. NMR Biomed 20:578–90.

Optimization of joint label fusion algorithm for automatic segmentation of prostate MRI

Fig. 1 Semi-automatic segmentation starts by (a) creating a ROI inside myocardium in which (b) k-means clustering is applied (b) to acquire a mask (c) for active snake contouring (d). Radial segments with expanding pixel layers were applied on the result (e) where CoV assessment (f), k-means clustering (g) and a correction to eliminate layers in radial segment two pixels wider that its neighbour were applied resulting in a myocardial mask (h). After adapting the thickness of the myocardial mask to the ‘‘true’’ myocardial area (i), the myocardial contours were acquired (j) Results Using composite images [1], two radiologists drew myocardial contours twice and generated the contours by semi-automatic segmentation as described above. On all slices, as compared to manual drawing, myocardial contours by semi-automatic segmentation acquired higher intraobserver and interobserver reproducibility, as expressed by the higher dice similarity coefficients (DSC) values (0.86 ± 0.05 vs. 0.79 ± 0.04, P \ 0.001 and 0.86 ± 0.04 vs. 0.76 ± 0.04, P \ 0.001 respectively). Pixel-wise T2* quantification [4] of the myocardial area segmented according to the American Heart Association 16-segment

123

Y. H. Choi1, J.- H. Kim2, H. J. Shin3, C. K. Kim2,3 1 Department of Health Sciences and Technology, SAIHST, Sungkyunkwan University, Seoul, South Korea 2 Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, South Korea 3 Department of Medical Device Management and Research, Samsung Medical Center, Sungkyunkwan University, Seoul, South Korea Keywords Joint label fusion  Multi-atlas segmentation  Prostate  MRI Purpose Recently, multi-atlas based segmentation approach has been proposed for automatic segmentation of human organs in MR and CT images to overcome the difficulty of the choice of the atlas in single atlas based segmentation approach. In multi-atlas based segmentation, the different atlases may produce similar label errors, because the weights were computed independently for each atlas. To address this limitation, joint label fusion (JLF) method was proposed to combine multiple atlases, in which weights were estimated to minimize the total expectation of labelling error considering pairwise dependency between atlases. In this study, we applied multi-atlas segmentation (MAS) with JLF algorithm into prostate MRI and the results according to different parameters were examined to optimize MAS with JLF algorithm for automatic segmentation of prostate MRI.

Int J CARS Methods (1) MAS with JLF algorithm In MAS procedure, segmentation of a target image FT is composed by registration and multiplication of a set of n atlases A1 = (F1,S1),…, An = (Fn,Sn). An ith atlas Ai is defined as a pair (Fi,Si) which denotes the registered atlas image and the corresponding segmentation (label). To minimize dependent segmentation errors between atlases, JLF proposes combining a pairwise dependency matrix with weighted voting label fusion process to find minimum total expected error [1]. In binary segmentation, segmentation errors can be modelled: ST ðxÞ ¼ Si ðxÞ þ di ðxÞ;

ð1Þ

where di(x) defines the label difference which exists between the ith atlas (Si(x)) and the target (ST(x)) at the position x. Since the label error occurs when the value of label difference is -1 or 1, the label difference can be described as a discrete random variable, characterized by the following distribution: qi ðxÞ ¼ pðjdi ðxÞj¼ 1jFT ; F1 ; . . .; Fn Þ

ð2Þ

JLF algorithm adopts weighted voting technique which assigns non-negative weight to each atlas to estimate consensus segmentation. Consensus segmentation in weighted voting is generated as SðxÞ ¼

n X

Wi ðxÞSi ðxÞ

ð3Þ

i¼1

and the weight sum is 1 [2]. With Eq. [1] and [3], total expected error can be found as follows. h i

Ed1ðxÞ;...;dnðxÞ ST ðxÞ  SðxÞ jFT ; F1 ; . . .; Fn 2 3 !2 n X Wi ðxÞdi ðxÞ jFT ; F1 ; . . .; Fn 5 ¼ Ed1ðxÞ;...;dnðxÞ 4

and used as the gold standard. First, the affine registration was performed to spatially transform multi-atlas MR images into a target MR image. From 14 atlas MR images, weighted voting was performed based on similarity between intensity of a target MR image and that of atlas MR images to segment prostate area. Joint label fusion method was used for the weighted voting to reduce the segmenting errors by computing dependency error matrix between each atlas. From 15 MR images, leave-one-out test was performed for optimization of three parameters: r, the radius of the local appearance patch N used in similarity-based Mx estimation; rs, the radius of the local searching patch Ns used in refining registration errors, and b, the parameter used to transfer image similarities in the pairwise joint label difference term. The range of each parameter was r[{3,4,5}, rs[{0,1,2,3}, b[{1,2,3,4,5,6}. For each case, the Dice similarity coefficient between automatic segmented prostate area and gold standard was computed. Results For automatic segmentation, 14 MR images were spatially transformed into a target image. Figure 1(a) showed overlap images for 14 MR images in a target image space. From 14 atlas MR images, weights were computed to minimize the total expectation of labelling error considering pairwise dependency between atlases. Figure 1(b) and Fig. 1(c) showed the posterior probability maps for nonprostate voxels and prostate voxels. The prostate was automatically segmented based on the posterior probability maps. Figure 1(d) showed the results of automatic segmentation of prostate in MR images. The volume of prostate region-of-interest for manually defined and automatically segmented methods was 32360 ± 5083 and 31959 ± 4333 mm3, respectively, which was not statistically different. Dice similarity coefficient values for each parameter (r,rs,and b) were shown in Fig. 2. The best dice similarity coefficient was 0.9034 ± 0.0377 for prostate segmentation with JLF with the optimal parameters; r = 5, rs = 1 and b = 2.

i¼1

¼

n X n X

  Wi ðxÞWj ðxÞEdiðxÞ EdjðxÞ di ðxÞdj ðxÞjFT ; F1 ; . . .; Fn

i¼1 j¼1

¼ Wtx Mx Wx ;

ð4Þ

where Wx is the set of voting weights [W1(x);…;Wn(x)], t indicates for transpose, and Mx(i,j), called a pairwise dependency matrix from atlas i and j, is estimated from intensity similarity.     Mx ði; jÞ ¼ p di ð xÞdj ð xÞ ¼ 1fFT ð yÞ; Fi ð yÞ; Fj ð yÞy 2 N ð xÞg 2 3b X   /4 ð5Þ jFT ð yÞ  Fi ð yÞjFT ð yÞ  Fj ð yÞ5 ; y2N ðxÞ

where N(x) presents a cubical neighbourhood around x and the size of local appearance patch is determined by patch radius r, and b indicates controlling parameter for weight distribution to estimate Mx. By using pre-determined dependency matrix Mx and Lagrange multipliers, Wx that minimizes total expected error can be acquired. (2) Registration errors refinement In the JLF algorithm, registration errors can be revised by finding point x’ which shows better intensity similarity in ith registered atlas image patch Fi[N(x’)] against the patch Fi[N(x)]. A search radius rs of cubical x’ search patch Ns is applied for finding the most similar image patch Fi[N(x’)]. The local search correspondence map between ith atlas and the target images can be described as follows: 2

ei ðxÞ ¼ arg minjjFi ðNðx0 ÞÞ  FT ðNðxÞÞjj

Fig. 1 The procedure of prostate segmentation with JLF. (a) The overlapped image of the registered atlas MRI. Posterior probability maps of voxels for non-prostate (b) and prostate (c). (d) Final segmentation. Green = gold standard prostate. Orange = automatically segmented prostate. Red = Overlap region

ð6Þ

x0 2Ns ðxÞ

In this study, the MAS with JLF algorithm was implemented using MATLAB (version 8.3. R2014a; The MathWorks, Natick, MA) (3) Experiments Fifteen healthy subjects underwent T2 weighted MR imaging. For each subject, the prostate area was manually defined

Fig. 2 Optimal label fusion parameter selection. (a) The mean of dice similarity coefficient against rs and r. The b is fixed at its optimal

123

Int J CARS value. (b) The mean of dice similarity coefficient against b and rs. The r is fixed at its optimal value Conclusion In this study, the MAS with JLF algorithm showed the best results with r = 5, rs = 1 and b = 2, suggesting the feasibility of join label fusion method for automatic segmentation of prostate MR image. For future work, the multi-atlas based segmentation with join label fusion algorithm was validated for automatic segmentation of prostate MRI with a large number of atlases using multiple measurements of similarity. References [1] Wang H, Suh JW, Das SR, Pluta JB, Craige C and Yushkevich PA, ‘‘Multi-Atlas Segmentation with Joint Label Fusion’’, IEEE Trans Pattern Anal Mach Intell. 2013 Mar;35(3):611–23. doi: 10.1109/TPAMI. 2012.143. Epub 2012 Jun 26. [2] Artaechevarria X, Munoz-Barrutia A, Ortiz de Solorzano C. Combination strategies in multi-atlas image segmentation: Application to brain MR data. IEEE TMI. 2009; 28(8):1266–1277.

Segmentation of tissue with the application of textural information J. Auyeung1, B. Chan2, A. Ibrahim2, G. Venne1, S. Pang1, J. Rudan3, G. Fichtinger2, M. Kunz2 1 Queen’s University, Department of Biomedical and Molecular Sciences, Kingston, Canada 2 Queen’s University, School of Computing, Kingston, Canada 3 Queen’s University, Department of Surgery, Kingston, Canada Keywords Intra-operative imaging  Segmentation  Tissue texture  Structured-light scanning Purpose Modern handheld structured light scanners have the ability to rapidly and non-invasively create 3d surface models with high accuracy and resolution. Due to a series of visible light pattern projected onto the object, 3d geometry of the object is determined and a surface model containing geometry as well as texture information is created (Fig. 1A). A scan of a mid-size object (such as the distal end of a femur) is normally obtained in 2–3 min without any need for calibration markers attached to the object. This makes structured light scanners perfect intraoperative image modalities for application in which only surface information of the object is required [1, 2].

Fig. 1 Segmentation of light-scanner model using texture information. A) Light-scanner model of knee specimen including texture information; B) Textural segmentation of muscle tissue using RGB value of (68, 41, 34) and tolerance value of 31 However, it is often required to identify specific anatomical regions in the scan, e.g. bony or cartilage part of the anatomy [1]. The manual selection of the region of interest intraoperatively can be time consuming and difficult to integrate into the surgical workflow.

123

The goal of this project was to investigate if texture information in structured-light scanned models can be used to aid in the identification of various tissue and support an automatic or semi-automatic segmentation of such surface models. Methods The study was performed on three fresh-frozen knee specimens. Two of the specimens originated from the same donor (knee #1, knee #2) and one specimen was collected from a different donor (knee #3). In each specimen four reference markers (screws) were rigidly attached to the bony part of the anatomy to aid in the alignment of different scans. Prior to scanning, the specimens were dissected in consecutive days to reveal different tissues. The following four dissection stages were investigated: 1. Muscle and tendon; 2. Ligaments, bone, and cartilage; 3. Bone and cartilage; and 4. Bone. After each dissection the specimens were scanned using a handheld structured light scanner Artec Spider (Artec Group, Luxembourg City, Luxembourg), which has a reported 3D resolution of 0.1, a 3D point accuracy of 0.05 mm, and a texture resolution of 1.3mp. All specimens were scanned in two different environments: a research laboratory in which the average brightness in the area of the specimen was 6170 lx (min: 5940, max: 6590); and in an operating room with surgical lights in operation, which resulted in an average brightness of 8920 lx (min: 6500, max: 12,000). Post-processing (including smoothing, noise reduction, mesh simplification and texture application) was performed using Artec Studio 10 Professional (Artec Group, Luxembourg City, Luxembourg). Models for each dissection state were aligned using the reference screws. To evaluate texture information and to perform textural segmentation, a custom-made extension module was developed for 3DSlicer. In each model the texture value for specific tissue was identified as a reference RGB (Red Green Blue) component value (r,g,b) and a color tolerance value t, which defined the range of texture values for the tissue with [(r - t,g - t,b - t), (r + t,g + t,b + t)]. To evaluate the deviation between two texture values, the Euclidian distance between the two reference RGB values was determined. Deviations between texture values for various environments, as well as for different specimens, were determined and analyzed. To establish the specificity for texture value for a tissue, manual segmented models were created for muscle, tendon, bone and cartilage. In addition, the light scanner models were segmented using the above defined texture values for each tissue. Deviations between the surface area of the manual and the textural segmented model were calculated and analyzed, see also Fig. 1. Results Our results showed that in average the texture value of tissue varied between the two different environments (laboratory versus OR) by 10.9. The tissue with the smallest deviations between the two rooms was muscle (average deviation of 6.9), and with an average deviation of 23.1 we found the texture value for tendons be most influenced by the change in environment. When we compared the tissue texture value for the two knee specimens from the same donor, we found an average deviation of 9.1, with the smallest average deviation for muscle (6.7) and the largest deviation for tendon (12.4). However, when we compared the texture values for specimens from different donors we found an average deviation of 78.3, with the smallest deviation for cartilage tissue (24.3) and the largest deviation for tendon tissue with 192.1. The comparison between the surface areas of manually segmented versus models segmented using the texture values, revealed the highest specificity for muscle with surface area deviation of 1,720 mm2. The surface area difference for the bone models was 6,325 mm2, for the cartilage 11,890 mm2 and the tissue with the lowest texture specificity was tendon with a deviation of 12,688 mm2.

Int J CARS

Conclusion The purpose of this pilot study was to analyze whether the textural components of a tissue could help with its segmentation in a structured-light scanner model of the anatomy. From the four investigated tissues, our results showed the most specific texture values for muscle tissue, which would suggest that texture value could aid in the automatic segmentation of this tissue. We found that cartilage and tendon presented with the least specific texture values. For these tissues, texture segmentation would only be reducing the area of interest, but not aid in a complete segmentation. Although our results showed similar texture values for tissue of specimens from the same donor, there were large deviations between the specimens originating from different donors. Unfortunately, the small sample size of this pilot study did not allow us to investigate the source of this deviation. Further studies with a larger sample size as well as different joint specimens would be beneficial to confirm if tissue texture is unique for individual patients. References [1] Rucker DC, Wu Y, Clements LW, Ondrake JE, Pheiffer TS, Simpson AL, Jarnagin WR, Miga MI (2014) A mechanics-based nonrigid registration method for liver surgery using sparse intraoperative data. IEEE Trans Med Imaging. 33(1):147–158. doi: 10.1109/TMI.2013.2283016. [2] Krempien R, Hoppe H, Kahrs L, Daeuber S, Schorr O, Eggers G, Bischof M, Munter MW, Debus J, Harms W (2007) Projector-based augmented reality for intuitive intraoperative guidance in image-guided 3D interstitial brachytherapy. Int J Radiation Oncology Biol Phys. 70(3):944–952. doi: 10.1016/j.ijrobp.2007.10.048

Fig. 1 illustration of the method. (a) Segmentation using Osirix. (b) Segmentation using StealthStation planning station (1) (2)

(3)

(4) Method to import DTI and fMRI series into stealthstation neuronavigation system R. Kamouni1, F. Schoovaerts1, N. Massager1, O. De Witte1 1 ULB Erasme Hospital, Neurosurgery, Brussels, Belgium Keywords Segmentation  StealthStation  DTI  fMRI Purpose There have been numerous advances in neurosurgery which have aided the neurosurgeon to achieve accurate removal of pathological tissue with minimal disruption of surrounding healthy neuronal matter. Advances including the development of neuronavigation systems. Additional imaging modalities such as DTI (for fiber tracking), and fMRI (for functional MRI) should now be incorporated into neuronavigational datasets in order to avoid damage to eloquent cortex and to avoid any neurological deficits resulting from surgery. Like other users of StealthStation neuronavigation system, we have met the problem to import DTI and FMRI imaging. This problem is one of the fact that a color palette is added to the blackwhite DICOM series. We present here validated method to overcome the problem of importing DTI and FMRI DICOM series in PLANNING STATION StealthStation ‘‘TREON plus’’ and ‘‘S7’’. Methods For our daily workflow we have implemented a server equipped with the open-source Osirix imaging software that stands as a miniPACS in-between the more rigid institutional PACS and the ‘‘StealthStation S7’’ planning station. Several open source packages like xMedcon, FSL, SPM, 3D-Slicer, Octave… are installed on this server. Our solution is mainly based on such software and could be replicated in 5 consecutive steps (Fig. 1):

(5)

Using the software Osirix as an image-processing tool to Convert RBG imaging to Black and White (B-W). Perform an ROI segmentation of fibers for DTI Dicom series and pads for FMRI with OsiriX and set the pixels inside the region to for example ‘‘1000 HU’’. Using a bash-script to modify the header and to batch convert a list of specific DICOM meta tags that are mandatory for StealthStations to accept the imaging without complaining. Sending back the imaging to Osirix for visual and conformance verifications. Transfer of the converted DICOM to StealthStation in order to proceed with the planning. As the fibers pixels have been set to a fixed value 1000 HU, the threshold tool will segment the fibers easily.

We created a user-friendly application that could be installed and configured to any server and perform these operations automatically on request. Results This application is used since more than 3 years for importation of DTI and FMRI series from many different radiological centers. Our protocol is 100 % robust, fast and versatile. This method does not alter the original imaging but simply ease the extraction of segmented fibers and the activated volume regions for the DTI and FMRI series respectively. Our protocol is 100 % robust, fast and versatile. This method does not alter the original imaging but simply ease the extraction of segmented fibers for the DTI. This method permits us to easily segment the activated volume regions on FMRI series Conclusion We create an application that modifies DICOM settings of DTI and FMRI series to conform to ‘‘StealthStation S7’’ standards. This solution has allowed us to easily incorporate fMRI activation volumes into our neuronavigational models for brain tumor surgeries to be helpful in ensuring a safe surgical trajectory and strategy for resection. DTI data can be simply added into navigational datasets to allow surgeons to plan and re-evaluate surgical trajectories in three dimensions and minimize the risk of surgical injury to white matter tracts.

123

Int J CARS An improved method for intrahepatic bile duct extraction from dual-energy CT volumes based on sample data adjustment for SVM training P. Chen1, H. Tanaka2, M. Oda1, Y. Hayashi3, T. Igami2, M. Nagino2, K. Mori1,3 1 Nagoya University, Graduate School of Information Science, Nagoya, Japan 2 Nagoya University, Graduate School of Medicine, Nagoya, Japan 3 Nagoya University, Strategy Office, Information and Communications, Nagoya, Japan Keywords Intrahepatic biliary duct extraction  Dual-energy CT  Abdominal contrast-enhanced  SVM Purpose To perform surgery with bile duct resection, it’s very important to know the specific spatial structure of the bile ducts and their relative locations with other organs in advance. Generally this is achieved through additional examinations using ERCP, MRCP, DIC-CT etc. However, as a result of some diseases such as biliary tract cancer, the bile duct may dilate and therefore becomes visible even in the general CT images. So, we tried to extract the bile duct automatically from CT images to reduce cost and alleviate suffering of patients due to additional examination. Koga el al. proposed some researches on this topic using single energy CT (SECT) images [1]. To obtain better bile duct segmentation approach, Chen el al. proposed a method for intrahepatic bile duct (IHBD) extraction from dual energy CT (DECT) volumes. They showed preliminary results of classifier-based IHBD extraction [2]. This paper presents an improved method for IHBD extraction based on adjustments of training samples utilized in the classifier training process. Methods (a) Overview—This paper denotes a CT volume at the lower energy level (100 kVp) as the volume A and the higher energy level (140 kVp) as the volume B. We extract a liver region from the DECT volumes. Firstly we train the SVM classifier to give a label of IHBD or non-IHBD for each voxel. The SVM classifier is trained by using training samples obtained from training cases. The training samples are chosen to show good performance in classification. In the test step, we extract the liver region from an input image to limit the processing area. Then the trained SVM classifier is utilized to classify a voxel inside the liver region into IHBD or non IHBD. (b) Training of classifier—We apply the median smoothing filter (3 9 3 9 3 voxels) to the training cases of DECT volumes. We segment liver regions semi-automatically from the smoothed DECT volumes by using the graph cut method. Also, we segment the IHBD regions manually. In the DECT volumes, IHBD regions are observed as regions that consist of low CT value voxels. Among voxels in the liver, differences of CT values between voxels in IHBD regions and voxels in non-IHBD regions are not large. Therefore, accurate segmentations of IHBD regions from DECT volumes are difficult, even if manual segmentations are performed carefully. For this reason, at edge of ground truth IHBD, even if the voxels are labeled as non-IHBD, there is a possibility that they actually belong to IHBD. We segment edge

123

regions of IHBD regions as described follows. We apply the dilation filter (t voxels) to the IHBD regions. The IHBD region is subtracted from the dilated IHBD region to obtain the edge region of IHBD. The edge region of IHBD is described as Re. We select training samples from training cases. We separate the liver region into three regions including: a) IHBD region, b) Re, and c) the remaining (outside of IHBD and Re) regions. The number of n training sample point are selected from the regions a) and c). For each training sample point, CT value and eigenvalues of the Hessian matrix for two scales r are computed as features. The Hessian matrix is calculated from a function that approximates a local CT values of a volume. Because we calculate feature values from two CT volumes of DECT, 14-dimensional feature vector is computed for each training sample point. Two-class SVM classifier is trained by using the feature vectors computed at selected sample points. The sample points located in the regions a) and c) are used as positive and negative samples, respectively. LIBSVM software library is used for implementing the desired classifier [3]. We use the RBF kernel for non-linear classification. The hyper-parameter C and c are determined by using the grid-search method that is also implemented in the LIBSVM. (c) IHBD segmentation—We apply the median smoothing filter (3 9 3 9 3 voxels) to input DECT volumes. Liver regions are semiautomatically segmented from the smoothed DECT volumes by using the graph cut method. We classify all voxels in the liver region into IHBD or non-IHBD by using the trained SVM classifier. The set of voxels that consists of voxels classified as IHBD are selected as the segmentation result. Results We applied the proposed method to two cases of DECT volumes. DECT volume specifications are 512 9 512 9 250 *350[voxels] and 0.5 * 0.7 9 0.5 * 0.7 9 0.8[mm] of voxel resolution. One case is utilized for training and the other (Case 2) is utilized for testing. The parameters were n = 5000, t = 1, and r = {1.0 mm, 2.0 mm}. The ground truth regions are shown in Fig. 1 and extraction results are shown in Fig. 2, respectively. For comparison, the best result of the previous method [2] is also shown in Fig. 2.

Fig. 1 Ground truth of Case 2

Int J CARS

Superpixels for the Visible Human Project and human anatomical voxel model construction G. Lee1, M. Bajger1, M. Caon2, G. Bibbo3, M. Ng4 1 Flinders University, School of Computer Science, Engineering and Mathematics, Adelaide, Australia 2 Flinders University, School of Health Sciences, Adelaide, Australia 3 Women’s and Children’s Hospital, Division of Medical Imaging, Adelaide, Australia 4 Hong Kong Baptist University, Department of Mathematics, Kowloon, Hong Kong Keywords Superpixel  Full body CT image segmentation  Visual human project  Human anatomical voxel model

Fig. 2 Extraction results of the proposed method (Up) and Chen [2] (Down) of Case 2 The evaluation was conducted manually by a surgeon who created the ground truth. He gave the following comments. (a) As for peripheral thin bile ducts, the extraction result is almost the same level of the ground truth, while the previous method [2] caused many FPs. (b) As for central thick bile ducts, the extraction result shows less over extraction. It becomes easier to recognize the confluence states of the bile ducts. The SVM classifier is unexpectedly robust to the imbalance of IHBD and non-IHBD training samples. In the Experiment A, the IHBD over non-IHBD ratio was almost 1:150. The classifier showed little sign of being negatively affected by the imbalance of samples. The over extraction was reduced considerably if we compare it with the result of the previous method [2]. Conclusion This paper presents an improved method for IHBD extraction based on adjustments of training samples utilized in the classifier training process. Training sample adjustments contributed to improve IHBD segmentation accuracy. As a future work, we will add more features to improve the IHBD segmentation accuracy and investigate the relationship between segmentation accuracy and samples utilized for classifier training. References [1] Koga K, Hayashi Y, Hirose T et al. (2015) Development of segmentation method of bile duct from abdominal CT volumes by using Support Vector Machine, Technical report of IEICE. MI, vol. 114, no. 482, pp. 227–232, (in Japanese). [2] Chen P, Tanaka H et al. (2016) Automatic extraction of intrahepatic bile duct based on features from dual-energy CT images, Technical report of IEICE. MI. [3] Chang CC, Lin CJ Libsvm home page. https://www.csie.ntu. edu.tw/*cjlin/libsvm/.

Purpose We have previously reported a Full Body Image segmentation tool, the FBIseg tool [1], for the Visible Human Project and similar projects around the world. The purpose of this paper is to present the novel superpixel approach for multi-organ multi-tissue segmentation in medical images and to demonstrate the efficacy and usefulness of the superpixel-FBIseg framework by segmenting the torso CT of a 14-year-old female and producing a human anatomical voxel model. Full-body image segmentation can be used in delineating the organs and tissues in the Visible Human Project and other similar projects. It is also the approach in constructing human anatomical voxel models from clinical volumetric imaging data such as CT or MRI for dosimetry calculation [2]. Though a key step in many research areas, literature shows that full-body segmentations were mostly manually performed with limited computer assistance. As such, it is notoriously laborious and time-consuming. An automatic system for full body segmentation would be desirable but has not yet been achieved. Machine segmentation has many advantages such as speed, reproducibility, pattern recognition and is not subject to human errors. However, it lacks expert knowledge. Our novel superpixel approach to full body segmentation reduces the large number of pixels in an image to a small number of superpixels. Expert knowledge is then incorporated with the use of the FBIseg tool [1]. Methods In recent years, superpixel has drawn increasing attention in computer vision. The application of superpixel to medical images is novel. Superpixel is the grouping of pixels into perceptually meaningful regions (truth objects) or part of the meaningful regions and thus can reduce computational complexity for subsequent processes. Different approaches can be used or adapted to generate superpixels. The wellknown K-means clustering method and its various modifications (e.g. [3]) and the state-of-the-art Statistical Region Merging technique [4] are just some examples. The criteria for generating appropriate superpixels may have different emphases according to the tasks. For segmentation tasks, it is important that each superpixel belongs to no more than one truth object (organ). The superpixel can cover an entire truth object or just part of a truth object. But, it should not contain more than one truth object or partially span several truth objects. In other words, a truth object (organ) can be the union of one or more superpixels. But the intersect of a superpixel and any truth object should be an empty set except for one truth object except for one truth object. For practical purposes, the underlying techniques employed in generating superpixels should be efficient computationally and in memory use. This is particularly so for the full-body segmentation problem at hand due to the large volume of data in CT and MRI and the large number of truth objects (organs/tissues). In this study, our choice of technique for the superpixel generation was Statistical Region Merging [4]. Using the FBIseg tool, the human expert identifies which organ/ tissue that each superpixel belongs to. The efficacy and usefulness of

123

Int J CARS the superpixel-FBIseg framework is demonstrated in the segmentation of a CT volumetric data set. A torso CT data of a 14 years old female was previously acquired and an anatomical voxel model, ADELAIDE was constructed manually in about one person-year time [4]. The same CT data (54 images of size 451 9 451 pixels at 1 cm intervals converted to 128 9 126 pixels) was re-segmented using the superpixel-FBIseg framework. Results Figures 1 and 2 show examples of the superpixel-FBIseg segmentation results. The new superpixel-FBIseg segmentation was performed by one of the authors MC (the same author performed the manual segmentation in [5]) in about two weeks. The accuracy of the segmentation using superpixels and the FBIseg tool was compared with manual segmentation [2]. The percentage differences in the number of voxels in corresponding organs/tissues were recorded. With reference to the manual segmentation [5], the percentage difference varies from good agreement (within 5 % in heart, liver, lungs, kidneys, combined bone and bone surface; and within 10 % in skin, spleen, breasts and muscle and soft tissues) to moderate disagreement (e.g. stomach 14 %, subcutaneous fat 16 %, oesophagus 18 %, uterus 26 %) and large disagreement (e.g. trachea 220 % and gall bladder 214 %). Moderate disagreements were mostly associated with walled-organs/ tissues while large disagreements were associated with small and/or walled organs with low image contrast. The partial volume problem in boundary voxels and the large surface to volume ratio in walled organs were observed as key factors in moderate and high disagreements. It is noted that, according to the theory of statistical region merging (the choice of method generating the superpixel), the probability for the generated superpixel containing more than one truth object (overmerging) is low. The above results suggested that overmerging was observed. The FBIseg tool does has built-in editing features that can edit the superpixels. This is at the expense of time and reproducibility.

Conclusion Using the superpixel-FIBseg framework, segmentation of 54 images of a torso CT data set of a 14-year-old female and the production of an anatomical voxel model reduced from over a year to about two weeks. This is a significant improvement in the efficiency in full-body segmentation and human anatomical voxel model generation. The accuracy of the segmented organs using the superpixel-FBIseg framework were compared to previous manual segmentation. Most of the organs segmented using the superpixel-FBIseg framework achieved good agreement with manual segmentation. Large disagreements are observed in small organ with low contrast such as the trachea or walled-organ with high surface to volume ratio such as the gall bladder. Inter-rater variability may also play a role and should be investigated. References [1] Lee G., Bajger M., and Caon M, FBIseg tool for the Visible Human Project, Int J CARS, 2014, Vol. 9(Suppl 1), pp. S40. [2] Caon M., Sedlar J., Bajger M. and Lee G., Computer Assisted Segmentation of CT Images by Statistical Region Merging for the Production of Voxel Models of Anatomy for CT Dosimetry, Australasian Physical and Engineering Science in Medicine, 2014, vol. 37, pp. 393. [3] Jing L., Ng M. and Huang J, An Entropy Weighting k-Means Algorithms for Subspace Clustering of High-Dimensional Sparse Data, IEEE Trans. Knowledge and Data Engineering, 19(8): 1026–1041, 2007. [4] Nock R and Nielsen F. Statistical Region Merging, IEEE Trans. PAMI, 2004, Vol. 26(11), pp. 1452. [5] Caon M., Bibbo G., and Pattison J., An EGS4-ready tomographic computational model of a 14-year-old female torso for calculating organ doses from CT examinations, Phys in Med. and Biol., 1999, Vol 44(9), pp. 2213.

Studying the aortoiliac morphological variability of patients with AAA based on cluster-specific templates F. Lalys1, C. Goksu1, S. Esneault1, A. Lucas2, A. Kaladji2 1 Therenva, RENNES, France 2 Pontchaillou University Hospital, Vascular surgery, Rennes, France Keywords AAA  Population-based analysis  Morphological variability  Aortoiliac anatomy

Fig. 1 An example of the superpixel-FBIseg segmentation (slice no. 48—at the top level of the liver). Left—superpixel-FBIseg segmentation (false colour for illustration purpose only); Right—original CT slice image

Fig. 2 An example of the superpixel-FBIseg segmentation (slice no. 68—abdominal region). Left—superpixel-FBIseg segmentation (false colour for illustration purpose only); Right—original CT slice image

123

Purpose Studying the morphological variability of a structure of interest is of importance for numerous applications in medical imaging. At present, most methods described to construct statistical shape models have similar procedures, aligning the subjects and performing statistical analysis. Statistical shape modelling has been mainly conducted for brain applications, often using statistical templates to enable voxelbased morphometry studies. In this context, application of principal component analysis (PCA) to the deformation fields has been widely employed trhough 3D statistical deformation models (SDMs) [1]. Even if construction of population-based models is a key issue in medical image analysis, it hasn’t gained much interest in abdominal applications yet due to the high morphological heterogeneity. We present here a general workflow that allows studying the anatomical variability of a very heterogeneous population based on clusterspecific templates, and we apply it to the aortoiliac anatomy within a population of patients with abdominal aortic aneurysm (AAA). Methods Patients and images—De-identification permits an operator to obtain computer tomography angiography (CTA) scans from 50 asymptomatic patients (57 ± 9 years old, 81 % male) originally intended

Int J CARS

for planning of EVAR. Scans were segmented using the EndoSize software [2]. Creation of the similarity matrix—The construction SDMs relies on the quality of the inter-subject registration. By definition, an average model cannot be created if non-rigid registrations do not converge, facing great challenge when presented to a heterogeneous population. Because of the high inter-subject anatomical variability of our population, we propose to use an intermediate step with cluster-specific templates following a hierarchical clustering approach. In an attempt to gather together patients with similar anatomy, we focused on the results of pair-wise non-linear registrations. Specifically, each possible combination of pair-wise registration from the 50 input images were computed. Then, a similarity metric based on the Dice coefficient allows the comparisons between co-registered images. Cluster analysis—Multidimensional scaling was first used to detect bad segmentations. Outliers were searched using k-means clustering in the 2D MDS space, and if necessary manually segmented. Then, an agglomerative hierarchical cluster analysis was performed on the similarity matrix using the Ward’s clustering linkage method. After gathering together patients with successful co-registrations, clusterspecific templates were created by iteratively testing the template creation workflow starting from the main cluster with all patients until achieving robust templates. Once created, these cluster-specific templates can finally be used as input volumes to create a unique template. Final deformation fields matching the input scans to the final template could be computed by merging both intermediate deformation fields. Results The 50 9 50 similarity matrix was computed from all pair-wise nonrigid registrations (Fig. 1). Large deformations were seen on iliac arteries and results showed that patients were mainly categorized based on the variation of this factor.

Fig. 1 Workflow for computing the similarity matrix Then, the hierarchical clustering was computed (Fig. 2). After moving the cut-off points at upper levels of the dendrogram, our hierarchical approach showed that the AAA population can be mainly divided into 5 anatomically different subtypes, each of the subtype ensuing one template. These 5 cluster-specific templates were successfully used to create a unique aortoiliac template.

Fig. 2 Dendrogram and final aortoiliac template PCA was finally employed to give an indication of the anatomical variability across the population and give insight into the underlying dimensionality of the data. The PCA technique is aimed at identifying the patterns of the input population and emphasizing their similarities and differences in term of size and shape variation. The mean shape and modes of shape variation showed that 22 principal components can describe [95 % of the total anatomical variability. Individual interpretation of modes of variations was however difficult. Using this methodology, the AAA population can be precisely describe to enhance our knowledge of the aorto-iliac anatomy and its numerous differences. Since PCA models the deformations as a linear combination of statistically uncorrelated principal components, new deformations can be created by changing the coefficients in the linear combination, and an infinite number of virtual shapes of the aorto-iliac anatomy can be modeled. Quantitative measures (e.g. neck length, renal arteries position, etc.) can also be directly taken on the final template or on sub-populations to better design stent-grafts. Rare anatomy could also be detected using this methodology and specifically analyzed to adapt the stent-graft shape design in order to better fit to the patient. Conclusion Not surprisingly, our analysis reveals the large inter-patient variability on the aorta and iliac arteries within the AAA population. For multiple applications around AAA, primarily in inter-population morphological analysis and stent-graft design, multi-subject templates of the aortoiliac structure are mandatory but have never been proposed so far. Our current methodology has been validated on a population of 50 patients, but further studies will include a large database of [500 patients in order to precisely describe the entire anatomical variation. References [1] Rueckert D, Frangi AF, Schnabel JA (2003) Automatic construction of 3-D statistical deformation models of the brain using nonrigid registration, IEEE Trans. Med. Imaging, vol. 22, no. 8, pp. 1014–1025. [2] Kaladji A, Lucas A, Kervio G, Haigron P, Cardon A (2010) Sizing for endovascular aneurysm repair: clinical evaluation of a new automated three-dimensional software, Ann. Vasc. Surg., vol. 24, no. 7, pp. 912–920.

123

Int J CARS NIRFAST-Slicer: open platform to accelerate the use of powerful image analysis and visualization tools in near-infrared optical imaging research A. Girault1, R. Ortiz1, S. C. Davis2, H. Dehghani3, A. Enquobahrie1 1 Kitware, Inc., Medical Computing, Carrboro, United States 2 Dartmouth College, Thayer School of Engineering, Hanover, United States 3 University of Birmingham, School of Computer Science, Birmingham, Great Britain

specific functions. This Matlab code is called through the MatlabCommander module, which first creates a connection to a Matlab server, then runs the execution of the code by specifying inputs set in the module interface inside NIRFAST-Slicer. It also converts the outputs of the Matlab execution back in the Slicer based application. A python module is packaged within NIRFAST-Slicer in order to read a mesh file created by NIRFAST-Matlab: it dynamically resamples the internal optical parameters recovered after the reconstruction phase into 3D volumes in the original image space. Those can then be conveniently overlaid above the original image.

Keywords Segmentation  Visualization and analysis  Optical imaging  Open source software Purpose NIRFAST is an interactive open source software application package that is widely used in near-infrared imaging research community [1]. It provides functionalities for modeling and reconstructing near-infrared light transport in tissue including standard single wavelength absorption and scatter, multi-wavelength spectrally constrained models and fluorescence models. It is used to model both light propagation in tissue (forward problem) and also recover internal optical parameters from optical data measured on the tissue surface (image reconstruction). To improve optical parameter estimation, anatomical medical imaging modalities have been used in NIRFAST. For this, NIRFAST has been using NIRView, a software package developed jointly by Dartmouth College and Kitware Inc., to create anatomical template [2]. However, the software platform upon which NIRView was developed was not easily extendable, making continued innovation and development of advanced image processing functionality challenging for non-expert users. As medical researchers continue to advance the field of NIR imaging by introducing increasingly sophisticated data types and analysis algorithms, higher data density, and novel application areas, there is a recognized need for a more flexible, extensible and widely-supported software application with advanced image analysis and visualization capabilities. In this paper, we introduce NIRFAST-Slicer, a 3D slicerbased application that provides powerful automated tissue segmentation tools that enable automated generation of complex FEM meshes from standard DICOM images, which are then used in the optical image reconstruction. Integration of 3D Slicer [3] with NIRFAST, brings a robust and widely used software platform for the analysis and visualization of medical imaging to the optical imaging research community. Methods NIRFAST is an interactive open source Matlab-based FEM package for modeling and reconstructing Near-Infrared light transport in tissue including single wavelength absorption and scatter, multiwavelength spectrally-constrained models and fluorescence models. 3D Slicer is a free, open source software platform for the analysis and visualization of medical images. It encapsulates cutting-edge segmentation and registration algorithms and provides support multi-modality imaging including MRI, CT, US, nuclear medicine, and microscopy. NIRFAST-Slicer brings together these two powerful open source toolkits. It provides powerful automated tissue segmentation tools that allow accurate FEM meshes to be developed and comes with a Matlab-bridge that facilitates data exchange between the Slicer platform and the NIRFAST Matlab package. The segmentation results from NIRFAST-Slicer are used to set up boundary conditions and internal tissue structure for the FEM analysis in NIRFAST-Matlab. In this paper, we present the software design and a case study. As shown in Fig. 1, the integration takes advantage of the 3D Slicer plugins mechanism. A few custom Matlab modules are packaged in NIRFAST-Slicer, composed of a XML description file defining the user interface through the Command Line Interface plugin mechanism, and a Matlab file that will call NIRFAST-Matlab

123

Fig. 1 NIRFAST-Slicer architecture Results NIRFAST-Slicer application has been used to investigate novel noninvasive MRI-coupled optical imaging approach to quantify receptor concentration and availability in sub-surface tumors in breast tissue. As shown in Fig. 2, the workflow involves: (1) Loading the subject MRI scan into the software application. (2) Creating a high quality visualization of the data. (3) Cropping the region of interest around the breast. (4) Placing the fiducials defining the light sources and detectors coordinates. (5) Creating a label map defining the boundaries for the mesh creation. (6) Creating the mesh using NIRFAST-Matlab functions. (7) Running the reconstruction process, and (8) Generating overlay visualization of the internal optical parameters.

Fig. 2 NIRFAST-Slicer workflow screenshots. Top-Left: 3D Rendering of the subject and Region of Interest placement. Top-Right: cropped volume visualization and light sources/detectors placement. Bottom-Left: multi-label segmentation and surface models rendering. Bottom-Right: overlay of an internal optical parameter in the original image space

Int J CARS Conclusion NIRFAST-Slicer is a 3D slicer-based application that provides powerful automated tissue segmentation tools that allow accurate FEM meshes to be developed and guide the recovery of optical parameters in the NIRFAST Matlab package. This integration brings 3D Slicer— a robust most widely used software platform for the analysis and visualization of medical imaging—to the optical imaging research community. References [1] Dehghani H, Eames ME, Yalavarthy PK, Davis SC, Srinivasan S, Carpenter CM, Pogue BW, Paulsen KD ‘‘Near infrared optical tomography using NIRFAST: Algorithm for numerical model and image reconstruction’’, Communications in Numerical Methods in Engineering, vol. 25, 711–732 (2009). [2] Jermyn M, Ghadyani H, Mastanduno MA, Turner W, Davis SC, Dehghani H, Pogue BW ‘‘Fast segmentation and high-quality three-dimensional volume mesh creation from medical images for diffuse optical tomography,’’ J. Biomed. Opt. 18 (8), 086007 (August 12, 2013), doi: 10.1117/1.JBO.18.8.086007. [3] Fedorov A, Beichel R, Kalpathy-Cramer J, Finet J, FillionRobin JC, Pujol S, Bauer C, Jennings D, Fennessy F, Sonka M, Buatti J, Aylward SR, Miller JV, Pieper S, Kikinis R ‘‘3D Slicer as an Image Computing Platform for the Quantitative Imaging Network’’. Magnetic Resonance Imaging. 2012 Nov;30(9):1323–41. PMID: 22770690.

Validation of the homology quantification: preliminary study

method

for

emphysema

M. Nishio1, K. Nakane2, Y. Tanaka3 1 Institute of Biomedical Research and Innovation, Kobe, Japan 2 Osaka University Graduate School of Medicine and Health Science, Department of Molecular Pathology, Suita, Japan 3 Chibune General Hospital, Department of Radiology, Osaka, Japan Keywords COPD  CT  Emphysema  Homology Purpose Chronic obstructive pulmonary disease (COPD) is characterized by chronic airflow limitation, which is usually progressive and associated with an inflammatory response of the lungs to noxious particles or gases. Technical advances in computed tomography (CT) facilitated evaluation of structural changes caused by COPD (emphysema and thickening of airway walls). Previous studies have shown that quantitative evaluation by CT is useful for assessing the severity of COPD. For emphysema quantification, percentage of low-attenuation lung area (LAA %) has been frequently determined by CT, and LAA % has been associated with COPD severity. However, no single type of quantification can guarantee an accurate assessment of emphysema. As for LAA %, because spatial distribution of low-attenuation lung regions was ignored in LAA %, it is expected that emphysema can be quantified more accurately by analyzing the spatial distribution of low-attenuation lung regions. For example, Fig. 1A and B shows representative images for LAA % = 30 % obtained from simulation study (in Fig. 1, white pixels represent normal lung regions, and black pixels represent low-attenuation lung regions). Although LAA % of Fig. 1A and B was the same, the spatial distribution of low-attenuation lung regions was apparently different between these images. To overcome this problem of LAA %, it is necessary to develop other types of emphysema quantification.

Fig. 1 Representative images of the binarized images at the percentage of low-attenuation lung area = 30 % obtained in the simulation study. (A) and (B) correspond to the results of simulations A and B, respectively. R of A and B is 0.103 and 0.380, respectively We hypothesized that homology could be used for evaluating the spatial distribution of low-attenuation lung regions, and that emphysema homology would be useful for determining the severity of emphysema. Visually, emphysema corresponds to the ‘‘holes’’ in the lung tissue. Originally, the concept of homology was that figures can be distinguished by their ‘‘holes.’’ Methods A simulation study was conducted to examine whether emphysema homology was associated with the spatial distribution of low-attenuation lung regions and investigate relationship between emphysema homology and LAA %. First, a binarized image was prepared in which each pixel could have two values: 0 and 1. Before the simulation started, the values of all pixels in the binarized images were 1. Then, in one simulation (simulation A), the pixel value was randomly replaced with 0. In the other simulation (simulation B), the pixel value was randomly replaced with 0 at a probability of 20 %, and the neighboring pixel value of the existing low-attenuation lung regions was replaced with 0 at a probability of 80 %. The replacement was continued until LAA % of the images reached predefined value, which ranged from 10 % to 50 %. After the replacement, the binarized images were evaluated to obtain the emphysema homology (b0, b1, and R). The relationship between R and LAA % was examined by plotting these values. Next, emphysema homology was evaluated in clinical cases. This study included 112 consecutive patients, and these patients underwent pulmonary function test and CT. Three CT images of the upper, middle, and lower lung fields were selected for each patient. LAA % was obtained using predefined threshold (-950 HU). Then, the CT images binarized by the predefined threshold were evaluated to obtain the emphysema homology (b0, b1, and R). In addition, b0 and b1 were normalized to the total number of lung pixels obtained from the three CT images and referred to as nb0 and nb1, respectively. The 112 patients were divided into three groups: Group A, nonsmokers; Group B, smokers without COPD, mild COPD, and moderate COPD; Group C, smokers with severe COPD and very severe COPD. The emphysema homology and LAA % were compared among these three groups. The differences in the emphysema homology and LAA % were tested by a t-test. Results In the simulation study, b0, b1, and R were different between simulations A and B even when LAA % was the same. Because the generation process of low-attenuation lung pixels was different

123

Int J CARS between simulations A and B, this result suggests that the differences in the spatial distribution of normal lung regions and low-attenuation lung regions were associated with those of the emphysema homology. According to Fig. 2, the relationships between R and LAA % were exponential and represented as R = 0.000307 x e0.193 x LAA % in simulation A and R = 0.0496 x e0.0682 x LAA % in simulatin B.

Methods The principle of N-wedge phantom is illustrated in Fig. 1. Suppose: pA denotes coordinates of p in coordinate system A; TA/B represents a homogeneous transformation that maps pB to pA; W is the world coordinate system defined in the tracking system; S is the sensor coordinate system defined in the sensor which is attached to the probe; P is the phantom coordinate system defined in the CAD model; I is the ultrasound image coordinate system.

Fig. 2 R and the percentage of low-attenuation lung area (LAA %) of the simulation study are plotted on semilog graphs. (A) and (B) correspond to the results of simulations A and B, respectively. The relationships between R and LAA % are represented as R = 0.000307 x e0.193 x LAA % in A and R = 0.0496 x e0.0682 x LAA % in B The emphysema quantification and results of t-tests among Groups A, B, and C show the following points; (1) between Groups A and B, the difference in only nb0 was statistically significant (pvalue = 0.00858); (2) in LAA %, b0, b1, nb1, and R, the differences between Groups A and B were not significant; (3) between Groups B and C, the differences in LAA %, b0, b1, nb1, and R were statistically significant (p-values B 0.01); and 4) in only nb0, the difference between Groups B and C was not significant. Conclusion Feasibility of emphysema homology was validated. Emphysema homology could be used for evaluating spatial distribution of lowattenuation lung region, and was associated with emphysema severity. Fig. 1 N-wedge phantom N-wedge phantom-based closed-form freehand ultrasound spatial calibration 1

1

2

2

2

X. Guo , Y. Lin , X. Zeng , H. Wang , F. Wang 1 Shanghai Jiao Tong University, Department of Mechanical Engineering, Shanghai, China 2 Shanghai Jiao Tong University, Department of Orthopedics, Shanghai, China Keywords N-wedge phantom  Closed-form  Freehand ultrasound  Calibration Purpose Freehand three-dimensional ultrasound has been widely used in many medical applications, such as cardiology, neurology, obstetrics and surgical navigation. The spatial calibration between the ultrasound image and the sensor coordinate systems is required in this technique, which is an important and prerequisite step in the aforementioned applications. Calibration methods based on N-wire phantom were discussed in several years [1], and almost all of these phantoms were manufactured with wires. In the ultrasound image it is challenging to accurately locate the actual intersection points of the ultrasound image plane and the wires [2]. However, N-wire phantom is still a compelling choice for ultrasound calibration because of its unique geometric structure and realizably automatic image segmentation algorithm [3]. In this study, the freehand ultrasound calibration system based on a novel N-wedge phantom is presented using an optical tracking system, aiming at improving the accuracy and convenience of the calibration procedure. The new phantom is not only the same principle of N-wire phantom, but also avoids the disadvantage of phantom made of wires.

123

The ultrasound calibration purpose is to find the transformation TS/I. The N-wedge phantom is consisted of five planes, named a, b, c, d, e from left to right, in which the planes a, b, d, e are parallel to each other and the plane c is intersected with the planes b, d. The coordinates of BP1 and C1P are known in the phantom CAD model. So the coordinates of QP1 can be calculated as QP1 ¼ ð1  kÞBP1 þ kC1P ;

ð1Þ

here k = ||QP1 -BP1 ||/||CP1 -BP1 || and the form |||| represents the Euclidean distance between two points. Because the line A1B1 is parallel to C1D1, the ratio could also be calculated as k = ||QP1 -MP1 ||/||NP1 -MP1 ||. At the same time, the intersection lines of the ultrasound image plane and five wedges in the phantom would display five gray-intensity lines in the image (the red lines in Fig. 1), which could be automatically segmented as five horizontal lines. In addition, two vertical lines are detected by the adjacent rectangles in the image. So the intersection points of three horizontal lines and two vertical lines are MIi, NIi, QIi, i = 1, 2. Bear in mind that the ratio k of the two distances is constant no matter which coordinate system they are in. Then it could be also calculated in the image coordinate system as k = ||QI1-MI1||/||NI1-MI1||. So the coordinates of QP1 can be calculated according to equation (1). With the same principle, the coordinates of QP2 are also acquired in this way. In the end, the calibration matrix are solved in the closed-form algorithm. Results The hardware configurations of the calibration system were the M-Turbo ultrasound machine (SonoSite, Seattle, USA), the L38xi 10-5 MHz linear-array ultrasound transducer (SonoSite, Seattle, USA), the Polaris Vicra optical tracking system (Northern Digital,

Int J CARS Waterloo, Canada), the ThinkPad X200 laptop (Lenovo, Beijing, China), the N-wedge phantom made of plexiglass, the image frame grabber T301 (TSW, Beijing, China) and the stylus, as shown in Fig. 2a. The software of calibration system was developed using some widely used toolkits in medical applications such as the VTK, ITK, IGSTK, OpenCV, Qt and Microsoft DirectShow. The software could simultaneously capture the image and record the location information of the ultrasound probe (Fig. 2b). Then the automatic image segmentation algorithm could find the intersection points of three horizontal and two vertical lines in the image (Fig. 2c). After capturing enough images, the calibration matrix could be calculated using closed-form algorithm. Mean point reconstruction accuracy (PRA) of the calibration results for 30 trials, each with 60 ultrasound images of the N-wedge phantom was 0.82 mm, which was sufficient to meet the requirement of medical applications. The average time of each calibration was less than 5 min.

Fig. 2 (a) Hardware configurations of the calibration system; (b) the graphical user interface of the calibration software; (c) automatic segmentation of the ultrasound image Conclusion In ultrasound calibration we have proposed a novel N-wire phantom variant called N-wedge phantom for the first time. For the conventional N-wire phantoms, the finite axial and lateral resolutions of the ultrasound transducer determine wire appearance in ultrasound image, and cause the small point-shaped objects being appeared as short lines in the image. Thus, it is a hard work to calculate the intersection points of the plane and the wires accurately. Compared with them, the new phantom takes advantage of the prosperity that the plane could be accurately detected as a line in the ultrasound image. The points used for calibration are determined by lines, not directly segmented from image. The ratio k could be accurately calculated in axial direction, which is better than lateral direction of N-wire phantom. The preliminary results have shown the accuracy of the system is enough for clinical applications and it is convenient to complete the calibration process. Besides, the phantom could be inexpensively manufactured, and it could be sterilized easily without any damage on the phantom. However, there is still a lot of work to do in the future research, such as the automatic accuracy feedback framework. References [1] Mercier L, Langø T, Lindseth F, et al. (2005) A review of calibration techniques for freehand 3-D ultrasound systems. Ultrasound in Medicine & Biology 31(4): 449–471. [2] Chen TK, Thurston AD, Ellis RE, et al. (2009) A real-time freehand ultrasound calibration system with automatic accuracy feedback and control. Ultrasound in Medicine & Biology 35(1): 79–93.

[3]

´ , et al. (2013) Improving N-wire Carbajal G, Lasso A, Go´mez A phantom-based freehand ultrasound calibration. International Journal of Computer Assisted Radiology and Surgery 8(6):1063–1072.

Molecular subtype evaluation for neo-adjuvant chemotherapy of breast cancer: dependent on ultrasonic vascularity features H.-L. Wang1, D.-R. Chen2, J.-Y. Huang1, Y.-L. Huang1 1 Tunghai University, Department of Computer Science, Taichung, Taiwan, Province Of China 2 Changhua Christian Hospital, Comprehensive Breast Cancer Center, Changhua,, Taiwan, Province Of China Keywords Breast ultrasound  Tumor vascularity, Neo-adjuvant chemotherapy  molecular subtypes Purpose Tumor vascularity, an important factor correlated with tumor malignancy, would be used to evaluate the effect of the neo-adjuvant chemotherapy prior to surgery and their correlation in the molecular subtypes of breast cancer. High-definition flow (HDF) Doppler ultrasound was performed to investigate blood flow and solid directional flow information in breast tumors. In this study,vascularity quantization and morphology features from HDF power Doppler ultrasound imaging were extracted as early predictors for evaluate chemotherapy effects. Firstly,this study designed an automatic method to extract vascular centre-lines from the tumor area. Then the vascularization indices were estimated from the extracted vascular centre-lines. Finally, the Chi square test, Student’s t-test ANOVA and support vector machine (SVM) model with all characteristics was employed to evaluate molecular subtypes for neo-adjuvant chemotherapy of breast cancer. Methods Data acquisition—This study recruited 72 consecutive T2 breast cancer (Tumor size [2cm and B5 cm) patients, who received neoadjuvant chemotherapy. The diagnosis of breast cancer was made by core needle biopsy. Pre-operative intravenous chemotherapy was given for six courses in each patient and three weeks per cycle. Epirubicin (Pharmorubicin, Pfizer Pharmaceuticals, New York City, NY, USA) 80–90 mg/m2, cyclophosphamide 500 mg/m2 and 5-Fluorouracil 500 mg/m2 on day 1 every three weeks. Sonographic examinations were done (period N1-N6) by using 3D power Doppler ultrasound with the HDF function (Voluson 730, GE Medical Systems, Zipf, Austria, equipped with RSP 6–12 transducer). The period N0 was the sonographic before the chemotherapy. The samples were classified into tumor subtypes based on inmuno-histochemical characteristics ofbreast cancer [1]: luminal, HER2+ and triple negative (TN). Vascular feature extraction—To quantify the vascularization of tumor, three fundamental indices vascularization index (VI), flow index (FI), vascularization flow index (VFI) were evaluated from 3D HDF Doppler ultrasound. Figure 1 shows an example of tumor region extraction procedure from the HDF Doppler ultrasound imaging. Moreover, this study performed 3D Gaussian low-pass filter to smooth the vascular images. After pre-processing, an automatic extracting method was performed to locate centre-line of each vessel [2]. The proposed method directly extracted vascular centre-lines from elongated 3D binary objects and provided good results and preserved topology. Figure 2 shows the vascular centre-line extraction procedure. Then seven vascular morphological features were also estimated from the extracted vascular centre-lines:

123

Int J CARS

Fig. 1 (a) Three-dimensional HDF power Doppler ultrasound imaging (GE Voluson 730 4D view) and (b) the proposed tumor region extraction procedure

(5) Number of tree (denoted NT); (6) Standard deviation of vascular direction (denoted DE); (7) Entropy of vascular direction (denoted EN); This study calculated the ten features from the tumor area and shell outside thickness 3 mm surrounding the breast lesion. Evaluation of chemotherapy response and tumor subtypes—The chemotherapy treatment effect of the 76 patients was evaluated by the clinical tumor response. The clinical tumor response was classified as complete response (CR), partial response (PR), stable disease (SD) or progressive disease (PD). The patients were classified into two groups: the responder, who was classified as CR or PR, and the nonresponder, who was classified as SD or PD. Among 76 patients, 57 were responders to chemotherapy, 19 remained stable in their disease or progressive disease. This study utilized SVM model to classify each patient as responder or non-responder. Moreover, the changes of vascularity features between chemotherapy stages were then utilized as the input to evaluate tumor subtypes of breast cancer. Results In this study, the simulation evaluated the vascularization indices from both the responder and non-responder groups to identify practical features for tumor angiogenesis. The results indicated that the features VI, VFI, MR, SD and NB would be potential early predictors for good responses to neo-adjuvant chemotherapy. These features were also valuable to evaluate molecular subtypes for neo-adjuvant chemotherapy of breast cancer. This work is helpful for patients to suffer less pain and spend less money by terminating subsequent useless chemotherapy. Conclusion This study evaluated vascularity features from the cases of the neoadjuvant chemotherapy using 3D HDF Doppler ultrasound imaging. Experimental results demonstrated that the feasibility of the proposed system for physicians to evaluate tumor subtypes of breast cancer and predict the effect o chemo therapy treatments. Early prediction of the effect of chemotherapy treatment could diminish the unnecessary chemotherapy treatments for patients. This is helpful for patients to suffer less pain and spend less money by terminating subsequent useless chemotherapy. Acknowledgements This work was supported by the Ministry of Science and Technology, Taiwan, Republic of China, under Grant MOST 104-2221-E-029-016. References [1] Ciriaa SC, Arago´na FJ, Mura CG et al. (2014) Magnetic resonance imaging in breast cancer treated with neoadjuvant chemotherapy: radiologic-pathologic correlation of the response and disease-free survival depending on molecular subtype. Radiologia 56(6):524–532. [2] Yen PL, Wu HK, Tseng HS, Kuo SJ, Huang YL, Chen HT and Chen DR (2012) Vascular morphologic information of threedimensional power Doppler ultrasound is valuable in the classification of breast lesions. Clinical Imaging 36(4): 267–271.

Development of HMD-based robot interface for throat surgery Fig. 2 Vascular centre-line extraction: the blue lines are the extracted vascular centre-lines, the red regions are blood vessels and the green area istumor’s region (1) Number of branch (denoted NB); (2) Shortest distance between vessels and the tumor center (denoted SD); (3) Maximum radian between vessels and the tumor center (denoted MR); (4) Variance of vessels (denoted VAR);

123

A. Lee1, B.-J. Yi1, J.-T. Seo1 Hanyang University, Ansan-si, South Korea

1

Keywords Head-mounted-display (HMD)  Master-slave robotic system  Throat surgery  User interface Purpose Recently, lots of applications have been developed using HMD device [1–4]. However, it has not been employed in the medical field for surgery. In order to provide a natural head posture to a doctor in the surgical environment, the HMD will be employed in this study.

Int J CARS Unlike previous surgical environment that watches the monitor, HMD offers a wider viewing angle so that it can provide the user with immersive feeling of view. The purpose of this study is to propose a new master interface system which uses the HMD device for comfortable operating posture and easy to control the endoscope position. In order to verify the usefulness of the HMD based interface system, a new 2-DOF endoscope holder was designed and integrated to a master-slave robotic system for throat surgery. Methods In typical laparoscopic surgery, the endoscope has to change its position to see the target legion. In order to see exact view, it is necessary to move the endoscope up and down for pitch motion and left and right for yaw motion. So an endoscope holder was designed to have 2 degrees of freedom. The newly designed endoscope holder can be controlled according to the measured user’s head pose angles. In order to display an endoscope image at HMD, the master PC has to get images from the endoscope console. So the image capture board was installed on the master PC. Then the master PC can get the stereoscopic live video by a webcam type camera. Then, HMD displays a different image to each eye to create an immersive image environment. An image capture board was used to send a stereoscopic image from the endoscope to the master PC. After image processing, HMD displays images to each lens. Then HMD displays the image of the target legion and the endeffectors attached at the distal end of the slave robot. So the user can feel an immersive experience. A phantom of human throat was tested using the new interface system with master-slave robot, which was developed for remote throat surgery. Figure 1 shows the flow chart of the HMDbased surgical interface system.

velocity as vectors up to a thousand times per a second. So the endoscope holder could be controlled almost real time. Thus, the user no longer has to move the computer mouse or hold the joystick. Even though its rendering delay is 20–30 ms, the user couldn’t feel any rendering delay.

Fig. 2 Operation test. (a) Experiment (b) Endoscope holder Conclusion This study aims at developing HMD-based interface system for dual arm slave robot used for throat surgery. Initially, the surgical procedure of endoscopic surgery was researched to analyze some requirements of interface system. Then, the endoscope holder was designed and the surgical interface system for throat surgery were developed and tested in this research. This approach can make much more intuitive surgical environment. Also, this system has some advantages over the conventional imaging systems, in aspect of reducing the user’s fatigue, decreasing the setup time for adjustment of endoscope, and feeling better sense of immersion. Nowadays, the HMD is continuously being developed, so it will have more reduced size and weight soon or later. Then the fatigue may be reduced when the user wears it. And the resolution of the display in the headset can also be enhanced. Thus it is expected to be improved to make it more immersive. References [1] http://www.engadget.com/2014/05/05/norway-tests-vr-in-armoredvehciles/. [2] http://imgur.com/a/srJbT. [3] Mathur AS (2015) Low Cost Virtual Reality for Medical Training. IEEE Virtual Reality Conference 2015 23–27 March, 978-1-4799-1727-3 [4] Haworth MB, Baljko M, Faloutsos P (2012) PhoVR: a virtual reality system to treat phobias. In Virtual-Reality Continuum and its Applications in Industry, pages 171–174, 2012.

Innovative utilization of IP addressable digital LED lighting in imaging environments

Fig. 1 HMD-based surgical interface system Results Figure 2 shows the operation test. The experimental environment is composed of a master robot, a slave robot, HMD, a stereoscopic endoscope, and a phantom of human throat. In experiment, the given task was controlling the endoscope holder‘s posture around the phantom throat by the user’s head pose control as well as displaying target legion on the HMD screen. HMD device includes a solid-state circuit that reports the device’s current acceleration and angular

M. M. Knopp1, M. M. Knopp2, K. Binzel2, J. Milacek2, C. Wright3, M. Knopp2 1 Emory University, Atlanta, United States 2 The Ohio State University, Wright Center of Innovation, Department of Radiology, Columbus, United States 3 The Ohio State University Wexner Medical Center, Department of Radiology, Columbus, United States Keywords LED lighting  Ambient lighting  Facility automatization  Visual guidance systems Purpose While dedicated commercial solutions for ambient lighting have been available for years and are increasingly being used in healthcare environments, broader utilization has been limited due to their high costs. In the last three years, digital addressable lights have entered broadly into

123

Int J CARS the consumer market and thus have become economical and available in a large variety of LED configurations. We explored and assessed feasibility of deploying such systems in imaging environments. Methods While there are several standards and approaches evolving from home automatization to bus based system technologies, we focused on one of the more broadly adapted standards of the ZigBee Alliance that has been developed with participation of the major lighting manufacturers. A major advantage of such a consortium’s standards is the potential interoperability of IP addressable LED lights and control devices from different manufacturers. Most manufacturers support dedicated or open source software development tools and have smart device apps available for local and remote control. Most devices are uniquely addressable and enable variation in intensity and light hue which can be programmed and are thus highly adaptable for innovative uses within imaging environments. We pursued the variability in hue for creating specific ambience such as a calming setting either in a patient preparation or injection room, the imaging system suites or reading rooms. We used replaceable bulbs in recessed housings, lightstrips and standalone fixtures. Patient, subject and staff experiences and visual perception of LED intensity and hue was recorded by interview and questionnaire. Results While we expected that adaptable light systems would be well received, we were pleasantly surprised how positive such environmental lighting approaches were received by all relevant parties. The benefit of light ambience in imaging environments were found to readily surpass the cost and efforts of deploying them. Consumer market based technology has advanced to the level of quality and robustness, that deployment in imaging/healthcare environments is feasible. While system manufacturers are frequently inclined to favor proprietary solutions, the open source community appears to provide enough incentives to support software development kits enabling innovative utilization. While an automated color/hue schema to match television images is a consumer application, adapting such concepts to provide visual feedback to guide patients appears powerful and effective for enabling the substitution or amplification of traditionally verbal commands. Conclusion Implementation of IP controlled digital LED lighting systems in imaging environments was found to be exceptionally well received. Consumer product grade LED systems appear to be of satisfactory quality and flexibility that much a broader use appears ready feasible. While standards and norms are still evolving, open source and consortium based software interfaces facilitate ready implementation of even sophisticated applications that can support visual directives or guidance as well as substantially improved room ambiance.

Double-blind validation studies indicate that surgical residents who received virtual simulator-based training outperformed residents without such training in performing complex procedures such as laparoscopic cholecystectomies [3]. iMSTK is an open source software toolkit designed for rapid prototyping of interactive simulation applications. It provides an easy to use framework that can be extended and can be interfaced with other third party libraries for the development of medical simulators without restrictive licenses. The framework is built using high quality software infrastructure and our goal is to cultivate a vibrant open source community around this framework. User communities will carry out experiments on iMSTK providing feedback and strengthening the toolkit. The framework will include resource management and real-time performance monitoring modules to provide an efficient and convenient test environment. Several open source frameworks for medical simulation currently exist. Among the most used is the Simulation Open Framework Architecture (SOFA), it provides an extensive framework and GUI mainly tailored to the research community for medical simulation and other applications in computer graphics and simulation in general. Another rising toolkit is OpenSurgSim, it is tailored to real-time medical simulation, it uses Open Scene Graph for its graphics engine and uses a module based design to add functionality to classes. There are also other frameworks and libraries that use physics based simulation such as ODE, Bullet, Havok but are mainly tailored to the game industry. Methods iMSTK is built to be extendable and provides interfaces and tools to help integrate external libraries. It has a modular design consisting of four main modules: Core, Rendering, Simulation, and Hardware, as shown in Fig. 1. These modules provide interfaces to well known external libraries responsible for the heavy lifting of the simulations, interaction and visualization. iMSTK also will provide in-house implementations of algorithms such as continuous collision detection and position based dynamics.

iMSTK: an open source interactive medical simulation toolkit R. Ortiz1, S. Arikatla1, T. Halic2, S. Radigan3, A. Girault1, S. De3, A. Enquobaharie1 1 Kitware, Inc., Medical Computing, Carrboro, United States 2 University of Central Arkansas, Computer Sciences, Conway, United States 3 Rensselaer Polytechnic Institute, Center for Modeling, Simulation and Imaging in Medicine, Troy, United States Keywords Interactive simulation  Toolkit  Open source  Surgical Purpose Medical simulators are powerful tools that assist in providing advanced training for complex procedures and objective skills assessment [1]. Simulators have the potential to accelerate training of residents without penalty of mortality, while improving skills where patient outcomes clearly correlate with surgical experience. This results in significantly fewer errors and shorter surgical times [2].

123

Fig. 1 iMSTK high level modules description The core module contains (because of this, the core module seems unsubstantial) interfaces needed to extend and integrate other libraries. The rendering module is responsible for creating complex visualization pipelines and relies in the well known and stable visualization toolkit (VTK) [4]. The visualization pipeline provides texture management, GLSL shaders support, and other advanced visual effects. The simulation module provides modules and interfaces to deal with the interaction of models in the scene and their

Int J CARS physics-based responses. It implements robust and stable time integration numerical algorithms to advance dynamics; collision detection and response methods to resolve contacts; and linear and non-linear numerical solvers. iMSTK relies on VegaFEM, an Open Source physics library for deformable model simulation. The hardware input/output module and interfaces supports over 30 devices through the Virtual Reality Peripheral Network (VRPN) external library [5]. iMSTK enables automated scene object and interaction management in order to reduce redundant and complex code writing for scene creation. The interface allows for the automatic creation and assignment of the collision detection, response assembly and solver to scene models. This is achieved by using a two stage method that traverses the model interaction graph describing the scene. iMSTK uses a quality control workflow that includes a stringent testing suite, continuous integration system (based on CTest and CDash), a web-based code review system, and a protocol for accepting patches into the main code repository. This entire infrastructure was setup in such a way that it is administered by the nascent community with minimal overhead. Results Testing and examples demonstrate some features of the toolkit, Fig. 2. The camera navigation example shows a simulation of the camera navigation task for laparoscopic surgery training. The shaders example shows how to create GLSL shaders for the visualization pipeline and the FEM simulation example shows a simple simulation of a soft body interacting with a rigid plane. iMSTK is also been internally used for several projects, one of such consist in creating a simulator for the dissection of Arteriovenous malformations (AVM) tumors as well as a simulator for the Fundamentals of Laparoscopic Surgery (FLS) training certification.

Conclusion We introduced an open source interactive medical simulation toolkit (iMSTK). Our toolkit is highly extendable and interfaces with well known and mature external libraries to accomplish the single goal of creating medical simulators as fast and painless as possible. It tries to accomplish this goal by leveraging external libraries for many of the heavy computations and algorithms and allowing the user to plug other libraries or its own external code with relative ease. This allows iMSTK to provide a solid foundation for creating simulation prototypes and at the same time a test bed for developing new algorithms. References [1] Satava RM (2007) Historical Review of Surgical Simulation—A Personal Perspective. World Journal of Surgery 32(2): 141–148. [2] Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA, Ramel S, Smith CD, Arvidsson D (2007) ‘‘Proficiency-based Virtual Reality Training Significantly Reduces the Error Rate for Residents During Their First 10 Laparoscopic Cholecystectomies.’’ American Journal of Surgery 193 (6): 797–804. [3] Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) ‘‘Virtual Reality Training Improves Operating Room Performance.’’ Annals of Surgery 236 (4): 458–464. [4] Schroeder W, Ken M, Lorensen B (2006) The Visualization Toolkit (4th ed.), Kitware, ISBN 978-1-930934-19-1. [5] Taylor II, Russell M, et al. ‘‘VRPN: a device-independent, network-transparent VR peripheral system.’’ Proceedings of the ACM symposium on Virtual reality software and technology. ACM, 2001.

Fig. 2 Examples providing some of the capabilities of iMSTK. (a.) Wet shader applied to a surface mesh using VTK shader support. (b.) Camera navigation scene simulation

123

Int J CARS

Poster Session 20th Annual Conference of the International Society for Computer Aided Surgery

123

Int J CARS The incidence of intraoperative MRI related adverse events during awake craniotomy K. Kamata1, N. Morioka1, N. Komayama1, H. Hasegawa1, A. Ohashi1, T. Maruyama2,3, Y. Muragaki2,3, M. Ozaki1 1 Tokyo Women’s Medical University, Anesthesiology, Tokyo, Japan 2 Tokyo Women’s Medical University, Neurosurgery, Tokyo, Japan 3 Tokyo Women’s Medical University, Faculty of Advanced TechnoSurgery, Institute of Advanced Biomedical Engineering and Science, Tokyo, Japan Keywords Intraoperative MRI  Awake craniotomy  Adverse event  Cardiorespiratory monitoring Purpose Intraoperative magnetic resonance imaging (iMRI) guided neurosurgery with functional mapping under awake craniotomy contributes to maximal surgical resection with minimal risk of postoperative deficits when a pathological lesion is in or adjacent to the eloquent area [1]. ‘‘Asleep-Awake-Asleep’’ technique is a kind of anesthetic method which is commonly used for awake craniotomy: that is, the patients are anesthetized until their craniums are opened, then they regain consciousness while functional mappings and surgical manipulation are preformed with free conversation, and finally they are under sedation during surgical site closure [2, 3]. Theoretically, invasive airway management, like endotracheal intubation or supraglottic airway use, is solely required during the initial ‘‘Asleep’’ phase. In some cases, psychological and physiological stress, such as emotional incontinence, epileptic attack, or nausea/vomiting caused by the awake craniotomy itself will trigger patient decline [2]. Additionally, an unsecured airway becomes critical while the patient is awake. Thus, careful preparation and patient observation are important for awake craniotomies. In general, iMRI is performed several times for each patient in order to confirm the updated surgical outcomes [4]. The magnetically active environment has disadvantages for patient safety management because MRI-compatible standard cardiorespiratory monitoring equipment is poorly developed. Direct observation of the patient is hard to do, as we should leave the patient alone in an MRI gantry. The aim of this study is to examine the incidence of intraoperative adverse events during iMRI, which were performed while the patient’s airway was not secured, as a part of awake craniotomy. Methods Anesthetic charts and surgical records of all awake craniotomy cases conducted at Tokyo Women’s Medical University were retrospectively reviewed. The sequences of iMRI scans performed without invasive airways were selected. General convulsive seizure, respiratory arrest, emotional incontinence, and nausea/vomiting were evaluated as critical events. The following cardiovascular parameter changes during iMRI were regarded as clinically significant: (1) hypertension with a more than 20 % increase of systolic blood pressure compared to pre-scan levels, (2) hypotension with a more than 20 % decrease of systolic blood pressure compared to pre-scan levels, (3) tachycardia with a more than 20 % increase of heart rate compared to pre-scan levels, and (4) bradycardia with a more than 20 % decrease of heart rate compared to pre-scan levels. Types of adopted cardiorespiratory monitoring were also examined. Results Between November 1999 and December 2015, a total of 371 awake craniotomies were carried out. The iMRI was used for 365 patients with 944 sequences, of which 580 sequences had pure oxygen supplied through a facial mask or nasal cannula. The average number and length of iMRI sequences without airway devices were 1.6 times per patient and 21 min, respectively. Critical events occurred in 21 patients with 24 sequences: general convulsive seizures occurred in 6 sequences; respiratory arrest

occurred in 2 sequences; nausea/vomiting occurred in 7 sequences; and emotional incontinence occurred in 9 sequences. The iMRI scan was emergently stopped due to patient decline in 4 cases; antiepileptic drugs were given to two seizure patients and invasive airway management was performed in the other 2 patients. Neither cardiac arrest nor accidental death occurred. Hyperdynamic changes were recorded in 90 sequences, of which 32 were hypertension and 58 were tachycardia. Hypotension and bradycardia were observed in 26 and 15 sequences, respectively. During iMRI, patient blood pressure, heart rate, and peripheral oxygen saturation were continuously monitored in 578, 578, and 555 sequences, respectively. Patient respiratory rate monitoring was poor and only used in 175 sequences. Conscious sedation was provided during 168 sequences with intravenous sedatives. Conclusion Remarkable technological advancement has increased the demand for iMRI-guided neurosurgery. Although sedation/anesthesia during diagnostic and therapeutic radiology is widely recognized, the iMRI-guided awake craniotomy is distinctive among radiological interventions because patient’s condition could be changed along with surgery. As availability and indications for iMRI expand in the neurosurgical field, the demand for awake craniotomy is expected to increase. Therefore, we examined the incidence of critical events during iMRI-guided awake surgery. Based on our 16-year experience, we consider that iMRI can be safely performed with careful observation even when a patient’s airways were not secured. Epileptic attack, nausea/vomiting, and emotional incontinence were major concerns. As general convulsive seizure and respiratory arrest were the reason for urgent discontinuation of iMRI, these common problems should be well controlled. If the patient’s condition was unstable, general anesthesia with invasive airway management should be selected. Monitoring the patient’s vital signs has been a frequent concern during MRI. It has been reported that the mortality risk doubled in an MRI environment [5]. As the urgent head rotation is complicated in craniotomies, respiratory rate monitoring is informative to detect a patient’s respiratory depression. Hypopnea becomes apparent in advance of desaturation. In fact, our cases of respiratory arrest were detected by a decreasing respiratory rate. The iMRI is useful for awake craniotomy if intraoperative complications are under control. Conventional cardiorespiratory monitoring should be attached, especially when the awake patient is left in the gantry without a secured airway. References [1] Black PM, Alexander E 3rd, Martin C, Moriarty T, Nabavi A, Wong TZ, Schwartz RB, Jolesz F (1999) Craniotomy for tumor treatment in an intraoperative magnetic resonance imaging unit. Neurosurgery 45(3):423–31. [2] Dinsmore J (2012) Challenges during anaesthesia for awake craniotomy, Essentials of neurosurgical anesthesia & critical care. Edited by Brambrink AM, Kirsch JR. 197–206. [3] Olsen KS (2008) The asleep-awake technique using propofolremifentanil anaesthesia for awake craniotomy for cerebral tumours. Eur J Anaesthesiol 25(8):662–9. [4] Muragaki Y, Iseki H, Maruyama T, Tanaka M, Shinohara C, Suzuki T, Yoshimitsu K, Ikuta S, Hayashi M, Chernov M, Hori T, Okada Y, Takakura K (2011) Information-guided surgical management of gliomas using low-field-strength intraoperative MRI. Acta Neurochir Suppl 109:67–72. [5] Girshin M, Shapiro V, Rhee A, Ginsberg S, Inchiosa MA Jr (2009) Increased risk of general anesthesia for high-risk patients undergoing magnetic resonance imaging. J Comput Assist Tomogr 33(2):312–5.

123

Int J CARS Gait characteristics and functional assessment of patients with flatfoot after arthroereisis

which indicates high decrese of the range of motion assymetry of the ankle joint movement in the dorsal-plantar plane.

N. Kapinski1, E. Lukasik2 1 University of Warsaw, Interdisciplinary Centre for Mathematical and Computational Modelling, Warsaw, Poland 2 The Jo´zef Piłsudski University of Physical Education, Warsaw, Poland Keywords Arthroereisis  Biomechanics  Flatfoot  STJ implant Purpose Flatfoot related issues are described as a lifestyle disease and cause significant costs of treatments. In Europe 6–10 % of the population suffers from the foot related issues which implies reduction of life quality, difficulty in daily functioning and force to break practicing sport activities, at both amateur and professional levels. The radiological characteristics in static conditions, under a load of weight shows that in the flatfoot the internal rotation of the talus is increased, which is connected with its excessive translation in the direction of plantar-medial-anterior in relation to the calcaneus and is the main reason of the gait pathology increase [1]. In order to restore normal characteristics of gait a surgical procedure (the Arthroereisis) is performed, which involves placing a motion blocking implant within the sinus tarsi, designed to restrict excessive subtalar joint (STJ) pronation while preserving supination. However limited studies on the STJ implant were performed which is the reason of insufficient knowledge regarding optimal anatomical design, suitable materials, biomechanical functions and even the general influence of the implant on a gait pattern. Thus within this work we present initial results and methodology used to acquire objective description of gait characteristics before and after implantation of the STJ implant. The knowledge will be further utilize in order to optimize design and material used for the implant production. Methods 10 patients with the flatfoot related issues were included in the study based on the defined inclusion criteria: hypermobility of the STJ found on both sides in the X-ray, no injuries and surgeries of the lower limbs, no pain in the lower limb during gait. Patients were measured with use of a motion laboratory situated in The Jo´zef Piłsudski University of Physical Education in Warsaw, equipped with the synchronized motion capture Vicon System, wireless EMG and three Kistler ground reaction forceplates. We registered three trials of the patients free gait. In order to calculate kinematics the vicon full body Plug-In-Gait protocol was utilized, while for the dynamics computations the GRF and EMG of chosen muscles were measured. The timestamps were defined to perform measurements before the arthroereisis as well as 3 months after the surgery, giving the time for an adequate rehabilitation. Results were analyzed with use of the standard OpenSim (musculoskeletal modelling software) workflow utilizing Gait2392 model i.e. scaling, inverse kinematics, inverse dynamics, RRA and CMC. Measurements defined in [2] were used to assess gait symmetry. Results Results are presented for the first patient who participated in all of the described studies. On Fig. 1 the values of the ankle angle (dorsalplantar plane) in time of the registration are presented. In order to assess the asymmetry the Assw was calculated defined as: Assw ¼ 2ðjðmaxðaL ðtssw ÞÞ  minðaL ðtssw ÞÞÞ  ðmaxðaR ðtssw ÞÞ  minðaR ðtssw ÞÞÞj=ððmaxðaL ðtssw ÞÞ  minðaL ðtssw ÞÞÞ þ ðmaxðaR ðtssw ÞÞ  minðaR ðtssw ÞÞÞÞÞ100% where max(a(tssw)) is the maximal value and min(a(tssw)) is the minimal value of the angle in swing phase. The exemplary result for the first gait cycle before the operation was 35.62 % and after 1.4 %

123

Fig. 1 Ankle angle values in dorsal-plantar plane in time of the registration before and after the arthroereisis On Fig. 2 the knee joint moments are presented. The first 30 % of the gait cycle on the chart represents the single support phase (SSP) where the STJ implant influence the gait pattern mostly. For both lower limbs the moments values decreased during the SSP respectively 25 % (right leg) and 87 % (left leg) comparing the values in time of the maximum moment before the surgery.

Fig. 2 Knee joint torques before and after the arthroereisis Conclusion The methodology of acquiring objective description of gait pattern characteristics of patients before and after Arthroereisis was presented. Described results indicate that the influence of the implantation of the STJ implant is significant, especially in the context of retrieving gait symmetry where for example the asymmetry of the range of the motion of the ankle joint could be decreased by more than 30 %. Performed studies have also shown that torques values in the main joints i.e. knee joint also were decreased which is important in order to stop the development of the pathology. Further work will be targeted on the detailed statistical analysis of the gathered data as well as mathematical modelling of the STJ region functional biomechanics. This will be done with use of the complex biomechanical models of the foot (e.g. developed based on 26 segments FootGM model [3]) and Finite Element Method analysis. References [1] Shibuya N. Characterisctics of Adult Flatfoot in the United States. The Journal of Foot & Ankle Surgery. 49: 363–368, 2010. [2] Kapin´ski N, Borucki B, Nowin´ski K: Error assessment and minimization in 4D motion tracking for functional orthopaedics diagnostics. Int J Comput Assist Radiol Surg 2013; 8(1): 157–159. [3] Sylvain et al. A new multisegmental foot model and marker protocol for accurate simulation of the foot biomechanics during walking, ISB 2011.

Int J CARS Fast autostereoscopic viewing system of medical image J. Li1, D. Wang1,2, W. Chu1, V. Mok3,4, L. Shi3,4 1 The Chinese University of Hong Kong, Dept. of Imaging and Interventional Radiology, Hong Kong, Hong Kong 2 CUHK Shenzhen Research Institute, Shenzhen, China 3 The Chinese University of Hong Kong, Department of Medicine and Therapeutics, Hong Kong, Hong Kong 4 The Chinese University of Hong Kong, Chow Yuk Ho Technology Center for Innovative Medicine, Hong Kong, Hong Kong Keywords Autostereoscopic monitor  Volume rendering  Viewpoints  GPU computing Purpose The autostereoscopic display of medical image aims to provide depth information for medical inspection. Conventional one-viewpoint monitor can display 3D medical images by employing the artificial information, (e.g., size, texture, transparency, and shadow) to distinguish depth between different structures. However, being different from that in the realistic 3D game or animation, these artificial highlights could not be arbitrarily applied to medical images which restore the true anatomical structures. Moreover, inappropriate change of size information could be useless or even misleading sometimes, especially in visualization of vessels. It is hard to judge whether vessels in small size are located in the further position or really small. Surgeons have to rotate and zoom the viewpoint in monitor frequently during surgery, in order to recognize relative positions of different tissues. Therefore, it is time consuming and confusing to use oneview monitor. Besides, due to lack of parallax, the conventional monitor cannot provide real 3D pictures. As another display system, the glasses type 3D monitors use the glasses to filter different angle pictures and send to different eyes. Whereas, in the circumstance of surgery, it is quite inconvenient for the surgeons to wear glasses. In comparison, the multi-viewpoints autostereoscopic monitor based on 9 different viewpoints could support clear 3D medical images from any position and possibly provide a glass-free display manner during surgery. Nevertheless, every autostereoscopic picture needs rendering extra 8 pictures, it is thus challenging to provide the real-time autostereoscopic medical images. For this, Magalha˜es et al. displayed 3D medical images in the autostereoscopic monitor by prerecorded video [1]. Portoni et al. used the segmentation results and texture reduced medical images to accelerate rendering [2]. However, these methods still fail to meet the requirement of real-time interaction and high fidelity of medical images in surgery. Therefore, in this study, we propose a GPU-based system for real-time autostereoscopic medical images view. Methods GPU based volume rendering—The volume rendering method is used to reconstruct the 3D image data and project it to a 2D plane. It is more time-consuming than surface rendering, but it can preserve more constructed specification of the original data. Therefore it is widely used in medical image visualization. One autostereoscopic picture is composed of different viewpoints images, and each viewpoint image need to be rendered respectively making the visualization process need huge computation time. For real-time rendering the autostereoscopic medical image, we use the OpenGL and GPU to accelerate the volume rendering. The GPUs containing hundreds of stream processors do well in float number computing and parallel computing. Compared with CPU, GPU is more suitable in the volume rendering which needs a lot of independent float computing. Multi-viewpoints interlacing—When watching the autostereoscopic monitor in different angles, two eyes can receive different viewpoints pictures by the refraction of lenticular lenses obliquely placed on the LCD. Based on the refraction theorem of light, Van et al. proposed a method to calculate the mapping relationship used to interlace the different viewpoints pictures into an autostereoscopic picture [3]. In the autostereoscopic viewing system, every image displayed on the

monitor consist of N viewpoints, so we should horizontally move the camera by N-1 times to get these pictures. Though larger adjacent viewpoints angle allows further views from monitor plane, it makes observers uncomfortable. We choose 1 degree adjacent angle which is a tradeoff of the stereoscopic effect and comfort. At last in mapping step, we assign different viewpoints pictures to the corresponding locations of the autostereoscopic picture. Results We develop the vtkstereoRenderWindow class which is based the multi-viewpoints mapping function to interlace the different viewpoints pictures into an autostereoscopic 3D image. The class has 4 parameters which are the dip angle between lenticular lenses and LCD, number of views per lenses, the viewpoints number and the resolution of the monitor. The class is easy-to-use on different lenticular lenses autostereoscopic monitors by changing these parameters. And this new class derives from vtkStereoRenderWindow can also be used to render the image into monitor. The vtkRenderWindow class could be used in any VTK based program by simply replacing the window class by it. Based on the GPU accelerating rendering and multi-viewpoints interlacing methods, we use our program to render a real CT data of cerebrovascular on an autostereoscopic 50 inches 4 K glasses-free 3D monitor (Chongqing Dromax Photoelectric Co. Ltd., Chongqing, China), which has a 3840*2160 resolution and 9 viewpoints. From the autostereoscopic image (Fig. 1) we could get a better understanding of the space position relations between the different vessels. The rendering autostereoscopic medical image can be smoothly rotated and zoomed and every viewpoint picture rendering time is about 0.04 s. We also test the one viewpoint rendering time under the CPU and GPU volume rendering methods, and the comparison result of different displaying resolution can be seen in the Fig. 2.

Fig. 1 Autostereoscopic monitor

Fig. 2 One viewpoint rendering time

123

Int J CARS Conclusion In this study, we constructed an autostereoscopic viewing system to realize a real-time multi-viewpoints volume rendering of unsegmented original 3D medical image data. Compared with other methods, it shows the advantage of glasses-free and can be simultaneously watched from different angles. With the help of this autostereoscopic system, the spatial information of different tissues is expected to be recognized quickly and accurately. Therefore, we hope proposed system could facilitate anatomical identification of tissue structures and thus benefit surgery in future. References [1] Magalha˜es D S F, Serra R L, Vannucci A L, et al. Glasses-free 3D viewing systems for medical imaging[J]. Optics & Laser Technology, 2012, 44(3): 650–655. [2] Portoni L, Patak A, Noirard P, et al. Real-time auto-stereoscopic visualization of 3D medical images[C]//Medical Imaging 2000. International Society for Optics and Photonics, 2000: 37–44. [3] Van Berkel C. Image preparation for 3D LCD[C]//Electronic Imaging’99. International Society for Optics and Photonics, 1999: 84–91.

Validation of a method for segmenting tumors in soft tissue applied to surgical planning C. Sua´rez-Mejı´as1, J. Pe´rez-Carrasco2, B. Acha2, J.L Lo´pez-Guerra3, T. Gomez-Cı´a4 and C. Serrano2 1 Technological Innovation Group, Virgen del Rocı´o University Hospital, Sevilla, Spain 2 Signal Theory and Communications Department, University of Seville, Sevilla, Spain 3 Oncology Unit, Virgen del Rocı´o University Hospital, Sevilla, Spain 4 Technological Innovation Group, Virgen del Rocı´o University Hospital, Sevilla, Spain Keywords Continuous convex relaxation  Retroperitoneal tumor  Surgical planning  Radiotherapy planner Purpose In 2005 an application called VirSSPA was designed and validated by different surgeons and engineers within the Virgen del Rocı´o University Hospital, Seville (Spain) for surgical planning. VirSSPA allows surgeons to generate 3D models using radiological images in DICOM format. VirSSPA is been successfully used in more than 1700 real cases, reducing the time spent in the operation rooms and in possible complications [1]. Nowadays, VirSSPA is used in the clinical routine to plan and simulate the surgical processes. However, the segmentation methods used by VirSSPA are not able to select tumors that appear in soft tissue. Actually, this difficulty is shared by most of the applications of surgical planning as well as radiotherapy planning workstation. The aim of this paper is presented an algorithm designed by authors for segmentation of retroperitoneal tumor and to offer a deep validation with others reference tools. Methods In this paper an algorithm which is based on the continuos convex relaxation [2] was designed by authors for segmentation of retroperitoneal tumors. To the best of our knowledge, only our previous works [3] have focused on the segmentation of retroperitoneal tumors. Convex relaxation optimization is used to minimize different energy terms. The proposed algorithm includes an accumulated gradient distance in the minimization step of the energy terms. For the validation of this algorithm a panel of experts was selected. The panel of experts was composed of clinicians with knowledge in

123

retroperitoneal tumor, two clinicians with expertise using VirSSPA and two clinicians with expertise using Pinnacle radiotherapy planning workstation. Then, the following experiments were carried out: (1)

(2)

(3)

Comparison with algorithms for segmentation implemented in VirSSPA software. Mainly, VirSSPA has three algorithms for segmentation. They are based on thresholding [4], region growing and adaptive region growing [5]. Two clinicians of the panel of experts with knowledge in VirSSPA segmented by consensus the tumors with differents algorithms implemented in VirSSPA. Comparison with algorithms for selection of tissue implemented in radiotherapy planning workstation. Concretely, Pinnacle was selected because is the machine that is used in the Virgen del Rocı´o University Hospital. In this experiment, two clinicians of the panel of experts with knowledge in this workstation, segmented by consensus the tumors with the algorithm implemented in Pinnacle. Comparison with designed algorithm. The tumors were segmented using the designed algorithm by authors of this paper.

Finally, the panel of experts manually contoured by consensus all cases and these segmentations were considered as ground truth. The results of each experiment were compared with the ground truth segmentations. Results The algorithm designed was implemented in Matlab language. 11 CT volumes with retroperitoneal tumors were selected of real patients from Virgen del Rocı´o University Hospital. Sensitivity, Specificity, Posotive Predictive Value (PPV) and Accuracy parameters were measured. In Table 1, the obtained results of the experiments are shown. The diferent algorithms implemented in VirSSPA provided similar results and the mean and standard desviation are represented in the Table 1. The proposed algorithm provides the best results. Table 1 Results (mean ± its standard deviation) obtained using algorithms included in VirSSPA, radiotherapy planning workstation and proposed algorithm respect to ground truth Algorithm

Sensitivity

Specificity

PPV

Accuracy

VirSSPA algorithms

0.13 ± 0.05 0.77 ± 0.02 0.68 ± 0.078 0.35 ± 0.023

Radiotherapy 0.11 ± 0.13 0.95 ± 0.03 0.40 ± 0.52 planning workstation

0.62 ± 0.51

Proposed algorithm

1.00 ± 0.00

0.90 ± 0.05 1.00 ± 0.00 0.84 ± 0.05

Conclusion A new algorithm based on continuos convex relaxation has been developed and validated for the semiautomatic selection of retroperitoneal tumors in CT images. It provides 90 % of sensitivity, 100 % of Specifity and Accuracy. The inclusion of this algorithm in software for surgical planing or in radiotherapy planning workstation allows to improve the selection of this kind of tumor. References [1] Gacto-Sa´nchez P, Sicilia-Castro D, Go´mez-Cı´a T, Lagares A, Collell M, Suarez C, Parra C, Infante-Cossı´o P, De la Higuera

Int J CARS

[2] [3]

[4] [5]

JM (2010). Use of a three-dimensional virtual reality model for preoperative imaging in DIEP flap breast reconstruction. J Surg Res 162: 1140–147. Continuous Max Flow at https://sites.google.com/site/www jingyuan/continuous-max-flow. Accessed 10 March 2016. Sua´rez-Mejı´as C, Pe´rez-Carrasco JA, Serrano C, Lo´pez-Guerra JL, Parra-Caldero´n C, Go´mez-Cı´a T, Acha B Three dimensional segmentation of retroperitoneal masses using continuous convex relaxation and accumulated gradient distance for radiotherapy planning, Medical & Biological Eng & Computing (MBEC), under revision. Gonzalez RC and Woods RE (2008) Digital Image Processing, Pearson Prentice Hall, Upper Saddle River, New Jersey Sa´nchez Mendoza C, Acha Pin˜ero B, Serrano Gotarredona M C, Go´mez Cı´a P T (2009) Self-Assessed Contrast-Maximizing Adaptive Region Growing. Lecture Notes in Computer Science 580:652–663.

A myocardium sheet transplantation robot system with cell sheet scooping mechanism and heart surface motion synchronization based on the image M. Takebayashi1, Y. Yamamoto1, R. Nakamura2 Chiba University, Department of Medical System Engineering, Chiba, Japan 2 Chiba University, Center for Frontier Medical Engineering, Chiba, Japan 1

Keywords Surgical robot  Heart surface motion  Motion synchronization  Myocardium transplant Purpose Myocardium sheets can restore the function of a failing myocardium; hence, they are presently used for the clinical treatment of heart disorders. However, handling fragile cell sheets without causing damage to their form or function is extremely difficult. In addition, off-pump treatment is required for a minimally invasive surgery. There is a device having a scooping mechanism that can be moved without breaking the cell sheets [1]. Nevertheless, using this device to transplant a cell sheet to a beating heart is still difficult. We have developed a heart surface motion cancellation and compensation robot system for myocardium sheet transplantation. This robot applies the scooping mechanism to its transplant mechanism [2]. Moreover, we use a pyramidal implementation of the LucasKanade (PLK) method for cardiac endoscopic image to present an analysis system for heart surface motion, including parallel, rotational, and scaling motions [3]. In this study, the analysis system was incorporated in the robot system. Moreover, we propose the robot system with compensating heart surface motion by measuring the heart surface motion and the synchronous motion control in two dimensions. Methods Figure 1 shows the configuration of the myocardium sheet transplantation robot system. The system comprised a three degrees-offreedom robot, an endoscope, and a personal computer. We used the optical measurement system (Polaris Vicra, NDI) and calibrated the endoscope coordinates in the robot. The heart surface motion was measured by the endoscope. Therefore, we defined the endoscope’s view as the robot’s control plane. The control plane was perpendicular to the endoscope’s cylinder axis.

Fig. 1 The myocardium sheet transplantation robot system First, we acquired heart surface images by using the endoscope. Second, we measured the feature points of the heart surface images by using the PLK method. However, the feature points gathered halation from the endoscope source. The robot’s tip (scooping mechanism) also appeared on the endoscope. Both events reduced the measurement accuracy. We removed the feature points that included these events to resolve the problem. We obtained the halation by using the threshold to analyze the halation. We then performed mask processing to extract the tip. Subsequently, we determined if the feature points were included in the halation and tip, and removed them accordingly. We predicted the heart surface motion by using a Fourier heart model to overcome delays during feature point measurement [4]. This initial Fourier heart model was created at the start of the measurement. The robot tracked the heart surface motion based on the Fourier heart model that was continuously updated by modification from the error between the prediction and measurement values. The robot worked in a direction perpendicular to the control plane while tracking the heart surface motion. The operator transplanted the myocardium sheet on the heart surface after contacting it. The transplantation was completed using these seps. Results We evaluated the robot system’s transplantation ability by experimenting on the myocardium sheet transplant. In this experiment, we inputted the heart surface motion of a pig to the XY stage. This heart surface motion was measured in the in vivo experiment. The robot then transplanted the myocardium sheet to the heart of the pig on the XY stage. The myocardium sheet phantom (circular: 14 mm diameter) was mainly composed of animal protein and liquid. For quantitative evaluation, we analyzed the sheet area ratio before and after the transplantation and the sheet shape retention rate. The experimental results showed that the transplantation success rate by using the robot system was 94.3 ± 9.4 % (n = 9), whereas that of manual transplantation was 75.2 ± 14.5 % (n = 9)[2]. The transplantation success rate by using the robot system was clearly higher than that of manual transplantation. Furthermore, the Mann– Whitney U test showed significant differences between the robot and manual transplantation procedures. Conclusion We developed a myocardium sheet transplantation robot system with a cell sheet scooping mechanism. The experiment showed that the robot system is more useful when compared to manual transplantation. However, wrinkles appeared thrice in the trial that was conducted 9 times. The robot system could not track the heart surface motion well because of a measurement error that occurred during heart surface measurement. Therefore, the measurement error should be decreased. In addition, the robot system could not transplant the sheet on the heart surface. Therefore, the tip is only operated when it is not in contact with the heart surface. Automating the transplantation operation in future work is considered.

123

Int J CARS

References [1] Tadakuma K, Tanaka N, Haraguchi Y, et al. (2013) A device for the rapid transfer/transplantation of living cell sheets with the absence of cell damage, Biomaterials, 34:9018–25 [2] Yamamoto Y, Takebayashi M, Nakamura R (2015) A surgical robot with cell sheet scooping mechanism and heart-surfacemotion synchronization control for myoblast cell sheet transplantation, ROBOMECH2015, 1A1-D07, in Japanese [3] Takahashi T, Hagiwara D, Nakamura R (2012) Heart surface motion analysis using endoscopic image for surgical robot of myocardial cell sheet implantation, Journal of Japan Society of Computer Aided Surgery, 14(3):212–3, in Japanese [4] Xu K, Nakamura R (2013) Development of the myocardium sheet transplant robot system using the heart surface motion prediction by the Fourier heart model, Journal of Japan Society of Computer Aided Surgery, 15(2):172–3, in Japanese

MRI-safe robot and novel workflow for MRI-guided pediatric long bone biopsy K. Cleary1, K. Sharma1, R. Monfaredi1, E. Wilson1, A. Krieger1, S. Fricke1, S. Lim2, C. Jun2, D. Petrisor2, D. Stoianovici2 1 Children’s National Health System, Washington, United States 2 Johns Hopkins University, Batlimore, United States Keywords Long bone biopsy  MRI-compatible  Needle guidance  Robotics Purpose Malignant bone cancers are the third most common pediatric solid tumors after lymphoma and brain cancers and include osteosarcoma and Ewing sarcoma, with thousands of cases in the US alone. Accurate histologic diagnosis is critical for the planning and initiation of surgery, chemotherapy, and or radiation therapy. Osteomyelitis is a bone infection, with over 50 % of reported cases seen in pre-school age children. Accurate diagnosis of the presence of bone infection and the infecting organism is critical for optimal therapy. Importantly, the imaging appearance of neoplastic and infectious pathology can be indistinguishable, making targeted and rapid tissue sampling key to clinical management. Typically a patient will present with symptoms including tenderness or reluctance to use the affected limb. Conventional radiographs may be normal. MRI is often used to aid in the diagnosis due to its improved soft tissue, marrow and joint space resolution. If a mass or area of marrow abnormality is seen on MRI, a biopsy will be ordered. The biopsy can be performed in the operating room by the orthopedic team, or under CT guidance in the Radiology suite, necessitating additional sedation/anesthesia and an open procedure in one situation, and exposure to ionizing radiation along with sedation/anesthesia in the latter. In both situations, significant additional time is required to achieve a definitive diagnosis. If the biopsy could be done immediately following imaging diagnosis of an abnormality in the MRI suite using robotic assistance as proposed here, trauma and radiation exposure to the patient could be minimized, precise sampling could be prescribed, and time to a final diagnosis could be significantly shortened. Methods We have developed a novel clinical workflow for robotically assisted bone biopsy in the MRI suite as shown in Fig. 1. The standard clinical workflow is shown on the left hand side, with the exception that the radiologist reviews the scan immediately while the patient is still on the table. If a biopsy is indicated, we move to the right hand side of the flowchart to proceed with an MRI-guided robot assisted biopsy.

123

Fig. 1 Novel clinical workflow for robotically assisted bone biopsy in the MRI suite An MRI -Safe (ASTM F2503) robot has been developed with 3 degrees of freedom (DoF). The robot orients a needle-guide about two orthogonal axes intersecting at a point located below the guide (Remote Center of Motion kinematics [1, 2]). The axes are actuated with two PneuStep motors [3] coaxially located on the robot base [4]. The needle is inserted manually through the guide. The depth of needle insertion is set by a third PneuStep motor located remotely. Before the insertion, this pre-adjusts the location of a depth stop along the barrel of the needle [4]. The robot is electricity free, uses air pressure for actuation, light for the position sensors, and is entirely made of nonconductive, nonmetallic, and nonmagnetic materials. Accordingly, the robot is MRISafe according to ASTM F2052, F2213, and F2182 based on the scientific rationale. The needle-guide, which comes in direct contact with the patient, is built of certified biocompatible material (ISO10993). The bore of the needle-guide can be made to accommodate various needles. The prototype was built for the MRI-Conditional Invivo 15100 bone biopsy needle (Invivo, Philips Healthcare, The Netherlands). Results The robot controller is a PC-based computer equipped with a motion control card. This is MR-Unsafe and should be located in the ACR (American College of Radiology) Zone III (control room or equipment room) [5]. An Interface controller including piezoelectric pneumatic valves and fiber optic sensors (magnetism free but electrically controlled) is located in the ACR Zone IV (scanner room) but outside the MRI scanner footprint. Bench tests of the robot have been completed for motion precision, accuracy, and structural stiffness. A Polaris optical tracker (NDI,

Int J CARS Canada) was used to measure the actual location of a passive marker placed on a rod attached to the needle-guide. Under careful measurement conditions, the accuracy of this optical tracker is as low as 0.055 mm. Experimental results showed an angular accuracy of 0.177 and a precision of 0.077. For a 50 mm deep target, the positioning accuracy is 0.155 mm and the precision is 0.067 mm. The stiffness of the mechanical structure has been measured with a force gauge and a micrometer which showed a structural stiffness of 34.5 N/mm at the needle-guide. The robot includes high contrast MRI markers for registration (filled with MR-Spots contrast, Beekley, Bristol, CT). A custom image-to-model registration algorithm and image-guided control software was developed. Initial tests were conducted in a Siemens MAGNETOM Tim4G scanner. Images of a gelatin mockup were acquired together with the robot. A 3D reconstruction (Fig. 2) shows a model of the robot registered to the image space, in which the model of the markers overlaps their images. These initial tests showed no apparent image artifacts or problems in operating the robot within the MRI.

[3]

[4]

[5]

Radiological Interventions. IEEE Transactions on Robotics and Automation. Oct 2003; Vol. 19(5) pp. 926–930.http://urobotics. urology.jhu.edu/pub/2003-stoianovici-ieeetra.pdf. Stoianovici D, Patriciu A, Mazilu D, Petrisor D, Kavoussi L: A New Type of Motor: Pneumatic Step Motor. IEEE/ASME Transactions on Mechatronics. Feb 1 2007; Vol.12(1) pp. 98–106. 2008 Best Paper Award of the Journal.http:// urobotics.urology.jhu.edu/pub/2007-stoianovici-tmech.pdf. PMC ID:21528106. Stoianovici D, Kim C, Srimathveeravalli G, Sebrecht P, Petrisor D, Coleman J, Solomon SB, Hricak H: MRI-Safe Robot for Endorectal Prostate Biopsy. Ieee-Asme Transactions on Mechatronics. Aug 2014; Vol. 19(4) pp. 1289–1299.http://urobotics. urology.jhu.edu/pub/2014-stoianovici-tmech.pdf PMCID:42194 18 ISI:000335915800019. Kanal E, Barkovich AJ, Bell C, Borgstede JP, Bradley WG, Jr., Froelich JW, Gilk T, Gimbel JR, Gosbee J, Kuhni-Kaminski E, Lester JW, Jr., Nyenhuis J, Parag Y, Schaefer DJ, SebekScoumis EA, Weinreb J, Zaremba LA, Wilcox P, Lucey L, Sass N: ACR guidance document for safe MR practices: 2007. AJR Am J Roentgenol. Jun 2007; Vol. 188(6) pp. 1447–1474. PMCID:17515363.

Compact forceps manipulator with spherical-coordinate linear and circular telescopic rail mechanism for laparoscopic surgery T. Kawai1, H. Hayashi1, A. Nishikawa2, Y. Nishizawa3, T. Nakamura4 1 Osaka Institute of Technology, Major in Biomedical Engineering, Osaka, Japan 2 Shinshu University, Faculty of Textile Science and Technology, Ueda, Japan 3 National Cancer Center Hospital East, Department of Surgical Oncology, Kashiwa, Japan 4 Kyoto University, Frontier Medical Sciences, Kyoto, Japan Keywords Surgical robot  Forceps manipulator  Laparoscopic surgery  Local operation

Fig. 2 3D Image reconstruction from MRI showing a gelatin mockup, robot registration markers, and robot model registered to the MRI space Conclusion In this abstract we presented a new clinical workflow for MRI guided bone biopsy and a new robotic assistant for the procedure. Experiments of the robot show that it is accurate, precise, and stiff. Comprehensive image-based targeting and image deterioration tests are in progress. The robot is MRI-Safe and preliminary imaging tests showed no mutual interference with the MRI. Our next steps are to evaluate the robot accuracy in MRI and then prepare an IRB application for a clinical trial. References [1] Stoianovici D, Whitcomb LL, Anderson JH, Taylor RH, Kavoussi LR: A modular surgical robotic system for image guided percutaneous procedures. Lecture Notes in Computer Science. 1998; Vol. 1496, pp. 404–410.http://urobotics.urology. jhu.edu/pub/1998-stoianovici-miccai.pdf. [2] Stoianovici D, Cleary K, Patriciu A, Mazilu D, Stanimir A, Craciunoiu N, Watson V, Kavoussi LR: AcuBot: A Robot for

Purpose Laparoscopic surgery that produces small scars has become widespread. When performing surgery through the incision, a surgeon must manipulate tools with limited degrees of freedom (DOFs) to minimize the effects of hand tremors and to cooperate with assistants using an endoscope or forceps. Master-slave remotely controlled manipulators such as the da Vinci systems have been developed to resolve these issues [1]. Because of possible emergencies, local operation in a sterilized area is safer than remote operation in a non-sterilized area. By integrating locally operated, small surgical robots and devices, a surgeon can perform safe, accurate, endoscopic, robotically-assisted surgery holding mechanical forceps with multiple DOFs with the hands stabilized by intelligent armrests while controlling an endoscope-holding robot with view stabilization and a forceps robot to grasp organs and provide traction [2]. We had developed SCARA and a mobile locally operated detachable end-effector manipulator (LODEM) as the forceps robot [3]. However, no locally operated compact forceps robot can provide a wide working area on abdominal wall. To this end, we propose a new compact forceps manipulator that can act as a third arm for the surgeon in the sterile environment of an operating room. For intuitive manipulation in a working area to avoid colliding with the doctors’ arms and the robot arm, the pivot point should be fixed mechanically, and the tool attached to the robot should not be guided by the fixed rail mechanism. The present study introduces a new compact LODEM that is a spherical-coordinate

123

Int J CARS manipulator with a linear and circular telescopic rail mechanism to make a wide working area on abdominal wall. Methods Figure 1 presents the photograph of the compact LODEM with 3 DOFs (yaw, pitch, and insertion/extraction) attached to a forceps. This prototype uses a circular telescopic rail with a linkage mechanism for the yaw and the pitch axes, and a linear telescopic rail for the insertion/extraction axis attached to a forceps. These telescopic rails consist of multiple circular sliders with 6 mechanical parts for the yaw and the pitch axes, and 3 mechanical parts for the insertion/extraction axis. Each axis is driven by a pair of cable rod covered with a coiled steel tube (1.2 9 4-mm-outer-diameter) attached to a stepper motor (3 Nm maximum torque). These three motors can be put on the table or attached to the bed rail. The compact LODEM can be removed from the surgical table as required, and the sterilized mechanism can be divided and the motors draped with a sterile cover. The operating range of the prototype is 0 to 90 for the yaw and pitch axes, and 0 to 150 mm for the insertion/extraction axis. The size is 180 mm 9 100 mm 9 90 mm when shortened and 230 mm 9 130 mm 9 120 mm when expanded. The multiple sliders were made of steel, and the mechanical parts including linkages weigh 250 g. The forceps is feed-forward controlled by a handheld interface of button switches.

(ii) Figure 2 presents the experimental setup. The organ model could be handled using the manipulator controlled by button switches. The surgeon could also pull the model organ in opposite directions using the forceps in the left hand and dissect it using scissors in the right hand. The specialist dissected the target organ smoothly.

Fig. 2 Experiment of simulated surgery performed by a specialist

Fig. 1 Prototype of compact LODEM for laparoscopic surgery The prototype was used in the following experiments for evaluation. (i) Motion trajectories of the telescopic rail mechanism for the three axes when the tip of the forceps was loaded by 0 N and 3 N were measured using an optical-displacement sensor (0.1-mm accuracy). (ii) Simulated surgery using the compact LODEM was performed on a surgically realistic gall bladder model in a training box. An endoscope and the manipulator were set in the training box. The operator, an endoscope specialist, used scissors in the right hand and forceps in the left hand. Results The results of the two experiments were as follows. (i) The maximum error between the hysteresis when loaded by 3 N was 1.9 mm for the yaw axis, 0.5 mm for the pitch axis and 1.7 mm for the insertion/extraction axis. The backlash of the telescopic rail defined as the horizontal maximum displacement when loaded by 0 N was 6.5 mm for the yaw axis, 1.0 mm for the pith axis and 3.4 mm for the insertion/ extraction axis. The mechanical deflection when expanded and loaded by 3 N was 1.8 mm for the yaw axis, 0.7 mm for the pith axis and 9.3 mm for the insertion/extraction axis.

123

The backlash and the deflection were caused by the assembling airgap for mechanical parts, but these were compensated by the trocar and grasping the target. In the simulated surgical procedures, the new manipulator could successfully handle the target while the arms of the specialist were not colliding with the manipulator. Conclusion We developed a feed-forward controlled, compact LODEM that can be used by a surgeon as a third arm during laparoscopic surgery. The cable rod driven linear and circular telescopic rail mechanisms are designed to facilitate minimally invasive, robotically assisted surgery by a doctor working near the patient. The results of the present study indicate that this device could be used for such applications. Future work will include in vivo simulated surgery performed by specialists. This work was supported in part by the Casio Science Promotion Foundation (2014-25) and JSPS Kakenhi (15K05917). References [1] Taylor RH, Stoianovici D (2003) Medical robotics in computer integrated surgery. IEEE Trans Robot Autom 19(5):765–781. [2] Kawai T, Matsumoto M, Horise Y, Nisihkawa A, Nishizawa Y, Nakamura T (2015) Flexible locally operated end-effector manipulator with actuator interchangeability for single-incision laparoscopic surgery. Int J CARS 10(Suppl 1):S246–S247. [3] Kawai T, Shin M, Nishizawa Y, Horise Y, Nisihkawa A, Nakamura T (2015) Mobile locally operated detachable endeffector manipulator for endoscopic surgery. Int J CARS 10(2):161–169.

Organ motion tracking system for laser surgical robot system in water-filled laparo-endoscopic surgery H. Nakata1, R. Nakamura2 1 Chiba University, Department of Medical System Engineering, Division of Artificial System, Graduate School of Engineering, Yayoicho, Inageku, Chiba, Japan 2 Chiba University, Chiba, Japan

Int J CARS

Keywords Laser surgery  Laparoscopic surgery  Medical robot  Motion tracking Purpose Laser surgery has been widely applied to medical treatments using pinpoint irradiation techniques and because of its ability to reduce bleeding. However, tissue damage through excessive heating and irradiation outside the target area is a drawback of laser surgery. Water-filled laparoendoscopic surgery (WaFLES) is a surgical technique that is performed in the abdominal cavity irrigated with liquid [1]. Using WaFLES, excessive heating could be reduced through the cooling effect of the isotonic solution in the abdominal cavity. However, it is difficult to perform laser irradiation accurately inside the abdomen because respiration will cause the abdominal organs to move and deform and their buoyancy during WaFLES may cause them to shift. In order to perform high-accurate and safe laser irradiation during WaFLES, we introduced a surgical robot system that uses an endoscopy video to measures organ motion inside the abdominal cavity filled with liquid [2]. Based on the previous results, we evaluated the irradiation accuracy of this robotic system while irradiating moving objects. Methods Figure 1 shows the configuration of an organ motion tracking system for a laser surgical robot system. First, we capture the image from the endoscope inserted into the abdominal cavity. Any image distortion due to the lens is then corrected using a camera calibration method developed by Chang [3]. Second, we determine the organ motion provided by the endoscopic image. To do this, we convert the image from camera coordinates to robot coordinates. The transformation matrix is calculated using four corresponding points before and after conversion. We then calculate the organ motion using an optical flow estimation method, which allows for high speed processing. Third, based on the amount of motion of the organ, we feed the control command to a laser surgical robot. Finally, the surgical laser irradiation can be performed precisely in accordance with the control command.

Fig. 1 Configuration of organ motion tracking system for laser surgical robot system Figure 2 shows the overview of the experimental test setup. To evaluate the irradiation accuracy of this system, a laser irradiates a 25 mm2 (5 mm 9 5 mm) area of the phantom moving in air and water. The laser irradiation begins through a key input from a PC. The phantom was a paper, which simulated the organ. The reference point used for tracking was placed in the upper left of the phantom. The phantom was moved by introducing a sine wave (amplitude: 2.5 mm, frequency: 0.25 Hz and 0.50 Hz) along the x- and y-axis. Laser irradiation was performed three times for each frequency and with a fixed phantom on a stage. The output of the laser was 2 W and 10 W in air and water, respectively. The irradiation accuracy was evaluated using similarity ratio (SR). SR is defined in Eq. (1) where T is the target area of the irradiation, and L is the actual irradiated area.

Fig. 2 Experimental test setup SR ¼ 100  2  ðT \ LÞ=ðT þ LÞ

ð1Þ

Results When the experiment is performed in air, the SR for a fixed phantom, moving phantom at 0.25 Hz, and moving phantom at 0.50 Hz are 90.0 ± 1.0,67.5 ± 1.5, 49.8 ± 2.2, respectively. When the experiment is performed in water, the SR for a fixed phantom, moving phantom at 0.25 Hz, and moving phantom at 0.50 Hz are 88.6 ± 2.1, 59.9 ± 2.8, 47.4 ± 0.3, respectively. From the results, SR in water is smaller than that in air. This is because the distortion correction has been insufficient in water. Conclusion We evaluated the irradiation accuracy of an organ motion tracking system for a laser surgical robot system operating in a WaFLES environment. This tracking system can be used to perform laser irradiation at an arbitrary time while tracking a target. The reduction of the SR when performing laser irradiation with moving objects may be due to the calibration error of endoscopic image, coordinate transformation error, tracking error of the optical flow estimation method, or tracking delay. In particular, the tracking delay seems to be the main factor affecting the result. Therefore, it is necessary to focus on tracking delay reduction in the future work. References [1] Igarashi T, Shimomura Y, Yamaguchi T, et al. (2012) ‘‘WaterFilled Laparoendoscopic Surgery (WaFLES):Feasibility Study in Porcine Model,’’ J Laparoendosc. Adv. Surg. Tech 22:Issue 1. [2] Nakata H, Nakamura R, Igarashi T (2015), ‘‘Organ Motion Tracking System for Laser Surgical Robot System Used in Water-Filled Laparo-Endoscopic Surgery—Development and Evaluation of Organ Motion Measurement System,’’ ROBOMECH2015, 1A1-D05, in Japanese. [3] Zhang Z (2000), ‘‘A Flexible New Technique for Camera Calibration,’’ IEEE Trans. Pattern Anal. and Mach. Intell., 22:1330–1334.

Reducing temperature elevation of bone drilling A. Feldmann1, J. Wandel2, P. Zysset1 1 University of Bern, Institute for surgical technology and biomechanics, Bern, Switzerland 2 Bern University of Applied Science, Institute for risks and extremes, Bern, Switzerland

123

Int J CARS Keywords Bone drilling  Robotic surgery  Thermal tissue damage  Robotic cochlear implantation Purpose Bone drilling is a surgical procedure which is used in many orthopedic, dental, otolaryngological (head and neck) and other surgeries involving the human skeleton. An important aspect of the related research concerns the production of heat during the drilling process [1] [2]. A previous in vivo sheep study [3] has shown that temperature elevation of minimally invasive robotic cochlear implantation can lead to thermal nerve damage in dense temporal bones. The aim of this study is therefore to find a safe drilling procedure for computer assisted surgeries like the minimally invasive robotic cochlear implantation surgery [4]. In this work, an extensive experimental study was conducted which focused on the investigation of three main measures to reduce temperature elevation as applied in industry: irrigation, interval drilling and drill bit design. Methods A new experimental setup (Fig. 1) was developed to measure drilling forces and torques as well as the 2D temperature field at any depth using a high resolution thermal camera. The advantage of a thermal camera compared to thermocouples is the ability to measure a whole temperature field. Additionally, there are no concerns of proper contact or precise placement of thermocouples within the bone. For these experiments, fresh frozen bovine tibiae of 4-year-old (milk) cows were used and acquired from a local slaughterhouse. Different external irrigation rates (0 ml/min, 15 ml/min, 30 ml/min), continuously drilled interval lengths (2 mm, 1 mm, 0.5 mm) as well as two drill bit designs (Ø2.5 mm) were tested. A custom drill bit was designed and manufactured with the aim of generating less heat. The drill bit was compared to a standard surgical drill bit (Synthes, Johnson&Johnson, USA). The drilling depth was 25 mm and the experiments were repeated 8 times for each parameter combination.

Fig. 1 Experimental setup for measuring temperature elevations, thrust forces and torques of bone drilling at any drilling depth. External irrigation system not shown in this Figure Results The results show that external irrigation is a main factor to reduce temperature elevation due not primarily to its effect on cooling but rather due to the prevention of drill bit clogging. During drilling, the clogging of bone material in the drill bit flutes result in excessive temperatures due to an increase in thrust forces and torques. Drilling in intervals (Fig. 2) allows the removal of bone chips and cleaning of flutes as well as cooling of the bone in-between intervals which limits the accumulation of heat. This also reduces the depth dependence of temperature elevation. However, reducing the length of the drilled interval was found only to be beneficial for temperature reduction

123

using the newly designed drill bit due to the improved cutting geometry. This finding was confirmed be the significantly lower thrust forces and torques of the newly developed drill bit in comparison with the standard surgical drill bit. To evaluate possible tissue damage caused by the generated heat increase, time and temperature elevation have to be evaluated [5]. Therefore, cumulative equivalent minutes (CEM43) were calculated and it was found that the combination of small interval length (0.5 mm), high irrigation rate (30 ml/min) and the newly designed drill bit was the only parameter combination which allowed drilling below the time-thermal threshold for tissue damage.

Fig. 2 Maximum temperature elevation for each parameter combining measurements at different drilling depths and repetitions in multiple samples. Influence of the three continuously drilled interval lengths on maximum temperature elevation Conclusion Temperature rise during bone drilling largely depends on the process parameters and the drill bit design [2]. Using no irrigation during drilling will lead to bone chip clogging the flutes and excessive temperature rise. Due to the low thermal conductivity of bone, accumulation of heat is a main concern which can be avoided when drilling in intervals. Additionally, the improved cutting geometry of the newly developed drill bit allows to drill with a lower temperature rise compared to the standard surgical drill bit. In conclusion, an improved drilling method has been found which could allow the drilling of more delicate procedures like the minimally invasive robotic cochlear implantation where nerves might be harmed by temperature elevation. The results also suggest that extensive tissue damage and bone necrosis cannot be avoided with current drill bits and process parameters as used in many surgeries. References [1] Labadie, Balachandran (2013) Minimally invasive image guided cochlear implantation surgery: First report of clinical implementation, The Laryngoscope 1–8. [2] Lee, Ozdoganlar, Rabin (2012) An experimental investigation on thermal exposure during bone drilling, Medical engineering & physics 34 (10) 1510–20. [3] Feldmann, Anso, Bell, Williamson, Gavaghan, Gerber, Rohrbach, Zysset (2015) Temperature Prediction Model for Bone Drilling Based on Density Distribution and in vivo Experiments for Minimally Invasive Robotic Cochlear Implantation, Annals of Biomedical Engineering (in press). [4] Bell, Stieger, Gerber, Caversaccio, Weber (2012) A selfdeveloped and constructer robot for minimally invasive cochlear implantation, Acta oto-laryngologica 132, 355–360.

Int J CARS [5]

Eriksson, Albrektsson (1984) The Effect of heat on bone regeneration: An experimental study in the rabbit using the bone growth chamber, Journal of Oral and Maxillofacial Surgery 42, 705–711.

Computer-aided planning of cranial 3D implants M. Gall1, X. Li2, X. Chen2, D. Schmalstieg1, J. Egger1,3 1 TU Graz, Institute for Computer Graphics and Vision, Graz, Austria 2 Shanghai Jiao Tong University, School of Mechanical Engineering, Shanghai, China 3 BioTechMed, Graz, Austria

user-friendly planning prototype. Our prototype was found easy to use and much faster. The prototype enabled planning a 3D implant in under thirty minutes by a surgeon, while the first author was able to obtain a similar result in twelve minutes. Figure 2 presents the result of a clinical case with a large cranial defect on the left side. The resulting output shows a well-fitting 3D implant on the inner as well as on the outer surface. The outcome of an additional Laplacian smoothing is shown on the right side of Fig. 2 (red circle). Tweaking on the vertex level is not necessary in our prototype, because markers can directly be placed on existing surfaces. However, individual markers may also be manipulated individually as fallback option, providing a higher degree of freedom.

Keywords Implants  Cranial  Planning  MeVisLab Purpose Computer-aided planning of cranial 3D implants has gained importance over the last decade due to the limited time in clinical routine, especially in emergency cases. However, state-of-the-art techniques are still very time consuming due to a low level approach. In general, CT scans are used to design an implant, often utilizing non-medical software, which is not really appropriate. Neurosurgeons spend hours with tedious low level operations on polygonal meshes for designing a satisfactory 3D implant. Commercial implant modeling software, like MIMICS, Biobuild or 3D-Doctor, is not always available, but if it is, using such software can be very complex. We present an alternative software allowing fast, semi-automatic planning of cranial 3D implants under MeVisLab. The method uses non-defected areas of the patient’s skull as a template for generating an aesthetic looking and well-fitting implant. This is done by mirroring the skull itself and fitting it, at least partly, in the defected area. Similar methods have been proposed in previous works [1], but our approach enhances the template with Laplacian smoothing [2], followed by Delaunay triangulation [3], to give the implant a significantly better fitting shaped. Methods For the cranial implant planning, a custom data-flow network was set up in MeVisLab [4], a popular medical imaging platform. In an first step, the patient’s dataset is loaded twice, whereby one object is mirrored to serve as a template (Fig. 1 left, green). Furthermore, for the planning process, it is necessary to have a view from the outside as well as from the inside onto the defected area. Therefore, we established a module network generating a cutting plane for both the original and the mirrored skull (Fig. 1 right). Then, markers are set, beginning with so-called edge markers for the borders of the implant. Next, we set the surface markers, which fit the mirrored object into the defect area. This is done for both sides, the inside and the outside edges and surfaces, to define the shape of the implant (Fig. 1 middle). The markers serve as input for the Delaunay triangulation module, available in MeVisLab. The triangulated mesh can be further enhanced by the user with a Laplacian smoothing module.

Fig. 2 Outcome of an additional Laplacian smoothing (red circle) Conclusion In this contribution, the semi-automatic planning of cranial 3D Implants under MeVisLab has been presented. We describe a MeVisLab prototype consisting of a customized data-flow network. Results shows that MeVisLab can be an alternative to complex commercial planning software which may also not be available in a clinic. There are several areas of future work, for example a comprehensive comparison and evaluation with commercial software products and an open-source version of our prototype [5]. Acknowledgement BioTechMed-Graz (‘‘Hardware accelerated intelligent medical imaging’’). Dr. Xiaojun Chen receives supports from Natural Science Foundation of China (81511130089), Shanghai Pujiang Talent Program (13PJD018), and Foundation of Science and Technology Commission of Shanghai Municipality (14441901002, 15510722200). References [1] Li X, et al. (2011) Symmetry and template guided completion of damaged skulls. Computers & Graphics 35, 885–893. [2] Amenta N, et al. (1999) Optimal Point Placement for Mesh Smoothing. Journal of Algorithms 30, 302–322. [3] Gopi M, et al. (2000) Surface reconstruction based on lower dimensional localized delaunay triangulation. Computer Graphics Forum, 19(3):467–478. [4] Egger J, et al. (2012) Integration of the OpenIGTLink Network Protocol for image guided therapy with the medical platform MeVisLab. Int J Med Robot, 8(3):282–90. [5] Li, X et al. (2016) A semi-automatic implant design method for cranial defect restoration. Computer Assisted Radiology and Surgery, PO 064.

A semi-automatic implant design method for cranial defect restoration Fig. 1 Overall planning workflow, left: original skull (white) and mirrored skull (green), middle: marker cloud with edge (green) and surface (magenta) markers, right: final implant Results On average, the overall planning time for a sufficient cranial 3D implant could be reduced to under thirty minutes, in comparison to three hours reported by our clinical partners. The mirroring of the skull, the setting of the markers and the smoothing is integrated in a

X. Li1, L. Xu1, Y. Zhu2, J. Egger3,4, X. Chen1 1 Shanghai Jiao Tong University, Mechanical Engineering, Shanghai, China 2 Huaiyin Institute of Technology, Jiangsu, China 3 Graz University of Technology, Computer Graphics and Vision, Graz, Austria 4 BioTechMed, Graz, Austria Keywords Cranial implant  Computer-aided design  Defect restoration  Model mirroring

123

Int J CARS Purpose The cranial defect restoration is required when a patient suffers from head trauma due to an accident or injury [1, 2]. However, the traditional method for the implant design is time consuming and complicated using industrial softwares such as Pro-Engineer (PTC, America), 3Matic (Materialise, Belgium), Geomagic Design (Geomagic, America), etc. [3, 4]. In this study, a semi-automatic implant design method for cranial defect restoration is presented, and a software named CranialCAD has been developed based on some well-known open-sourced toolkits including VTK (http://www.vtk.org/) and Qt (http://qt-project.org/). Aiming at evaluating the efficiency of our method, several case studies were conducted and the comparison with traditional ways was also discussed. Methods The workflow of our method is shown in Fig. 1, which is described as follows:

to reduce the overlapped points. An initial model is reconstructed using Delaunay triangulation algorithm through the points. 4. Generation of the implant: The outer surface of initial implant is generated by clipping the initial model using the contour in step 3. An isosurface S defined from Eq. (5) is formed as the inner surface using MC algorithm. S ¼ fðx; y; zÞ : f ðx; y; zÞ ¼ ag

ð5Þ

where f(x,y,z) is the signed distance and a is the thickness of the implant. Ruled surface is used to connect the inner and outer surface of the initial implant. The final implant is obtained using the same method using another contour drew from the inside of the initial implant along the edge of the defect. Results The CT data of a patient can be imported into CranialCAD to reconstruct the cranial model with defect based on image segmentation, region growing and marching cubes algorithm. First of all, a mirrored model is created according to the median plane, and then a surface is fitted on the basis of the region surrounding the skull defect, and the final implant is generated using surface clipping, surface offsetting, and ruled surface construction. On the basis of this method, the phantom experiments for four cases were conducted using the CranialCAD (shown in Fig. 2 (a)–(d)), resulting in anatomically wellfitting and highly symmetric implants. The comparison study for the case (d) was also carried out between the implant designed using CranialCAD and the result (shown in Fig. 2(e)) using the traditional softwares of UG, Geomagic and Materialise Magic RP together. The distance color map (shown in Fig. 2(f)) shows that there is no significant difference at the edge of the implant model between two models. As our method is based on the mirrored model, the implant designed using CranialCAD is more symmetric than the traditional way.

Fig. 1 The work flow of CranialCAD: (a) mirroring the model; (b) clipping the mirrored model; (c) Surface fitting and the generation of the initial implant; (d) the generation of the final implant; (e) (f) the final cranial implant and restoration effect; (g) the 3D printed implant model 1. Mirroring model: The median plane is generated for the mirroring according to the anatomically symmetric points of the skull model. Given N (x, y, z) is the normal of the plane, R is the vector of the rotation axis, and A is the rotation angle. The mirroring method is realized through a transform matrix T acquired from the equations blow: 1 T ¼ T1 T2 T3 T1 2 T1

ð1Þ

R ¼ cross     A ¼ a tan sqrt z2 þ y2 =x =p 180

ð3Þ

ð2Þ

where T1, T2, T3 are respectively the transform matrix for translating and rotating the model. 2. Clipping surface: A contour is indicated along the edge of the defect. An implicit function is constructed based on the signed distance field determined by the dot product of a vector V from Eq. (4) with the normal of vertex X. Using a vtk class named vtkClipPolydata, the mirrored model is clipped through the implicit function. V¼XM

ð4Þ

where M is the nearest point to X on the contour. 3. Surface fitting: Another contour is drawn around the defect region. The points inside the contour are projected on the screen to obtain the view coordinates and then converted into world coordinates

123

Fig. 2 (a) (b) (c) (d): Phantom experiments for four cases using CranialCAD; (e) the implant design using traditional softwares; (f) the distance comparison between the two implants of (d) and (e) In addition, only three manual steps of our method i.e., the median plane generation, and two contours’ drawing, are required, making it much easier and more efficient. Generally, it takes less than 30 min to accomplish the whole design using CranialCAD, whereas at least 2–3 h are needed when using traditional softwares. Conclusion In this study, a semi-automatic implant design method for cranial defect restoration was proposed. The phantom experiments validated the feasibility and efficiency of the CranialCAD. However, our method has limitation for a skull with large area defect near the temporal bone, as the surface fitting algorithm may fail since the surrounding surface is not convex. A similar workflow under Me VisLab showed great efficiency for the cranial implant planning, which can help us to improve the algotithm [5]. Nevertheless, future research still needs to be done and clinical trials aiming at evaluating the reliability of CranialCAD will be conducted. As the CranialCAD

Int J CARS is developed on the basis of open sourced toolkits, it will be released as an extension module of 3D Slicer (an open sourced software platform for visualization and medical image computing, www.slicer.org) in the near future so that it can be shared by the global community. Acknowledgment This project was supported by Natural Science Foundation of China (81511130089), Foundation of Science and Technology Commission of Shanghai Municipality (14441901002, 15510722200), and the Foundation of ‘‘Jiangsu Provincial Key Laboratory for Interventional Medical Devices’’(jr1411). Dr. Jan Egger received support from BioTechMed-Graz (‘‘Hardware accelerated intelligent medical imaging’’). References [1] Pompili A, et al. (1998) Cranioplasty performed with a new osteoconductive osteoinducing hydroxyapatite-derived material. JNeurosurg 89:236–242. [2] Hieu LC, et al. (2003) Design for medical rapid prototyping of cranioplasty implants. Rapid Prototyping Journal 9(3):175–186. [3] van Der Meer WJ, et al. (2013) Digital planning of cranial implants. British Journal of Oral & Maxillofacial Surgery 51(5):450–452. [4] Lethaus B, et al. (2011) A treatment algorithm for patients with large skull bone defects and first results. Journal of craniomaxillo-facial surgery : official publication of the European Association for Cranio-Maxillo-Facial Surgery 39(6):435–440. [5] Egger J, et al. (2016) Computer-Aided Planning of Cranial 3D Implants. Computer Assisted Radiology and Surgery, PO 177.

Full torso 3D structured light points cloud scan for personalized orthopaedic brace design and clinical application W. Glinkowski1,2, K. Mularczyk3, J. Michon´ski3, W. Sieteski4, 4 _ , W. Budacz4, K. Krawczak5, K. Walesiak5, K. Zmuda _ A. Zukowska5, R. Sitnik3 1 Baby Jesus Clinical Hospital, Orthopaedics and Traumatology of Locomotor System, Warszawa, Poland 2 Medical University of Warsaw, Medical Informatics and Telemedicine, Warszawa, Poland 3 Institute of Micromechanics and Photonics, Warsaw University of Technology, Warszawa, Poland 4 VIGO-Ortho Polska, Warszawa, Poland 5 Medical University of Warsaw, Chair and Department of Orthopaedics and Traumatology of Locomotor System, Warszawa, Poland Keywords Structured light  Scoliosis bracing  Personalized treatment  Points cloud scan

personalized and accurately fit the patient’s body surface of the trunk. Three-dimensional computerized methodologies may warrant the optimal brace manufacturing and improve treatment [2, 3]. The purpose of the study was to perform the field test in the clinical practice the four-dimensional structured light system for design and manufacturing custom, personalized brace. Methods Fifty-five were enrolled in the study. The mean age of patients was 13.6 years. The mean thoracic angle scoliosis was 30 degrees. Scoliotic patients enrolled in the study were scanned in the multidirectional body surface scanning laboratory, located at the Baby Jesus Clinical Hospital. The IRB approval has been obtained for clinical studies. Written informed consent was obtained from the parents/legal guardian of the patients. The 3D structured light scanning system consists of four uni-directional systems [4, 5]. Each system was calibrated in a common coordinate system and a PC computing unit. The camera recorded images (raster images and binary sinusoidal modulation of brightness) and processed by the scanner’s software. The 3D shape of the patient’s image of the torso surface was fast acquired. (approximately 1.5 s.). The 3D image was virtually combined from merged uni-directional scans. The precise 3D surface topography was supplemented with X-ray diagnostics only when necessary. The measurement data were obtained in the form of a point cloud further converted into a 3D model in STL format for further manufacture the orthosis. Point cloud accuracy is as high as less than 0.5 mm. Specified back and scoliosis Patient-Oriented Instruments were designed to collect data on quality of life assessment, including scoliosis, deformity, brace and disability questionnaires. Results Braces for the real AIS cases were manufactured accordingly to the methodology described. The 3D model was obtained for all patients using 3D four-dimensional structured light scanning. Point clouds were determined using Grey codes and sinus light fringes. Scans were transformed to the x, y, z coordination system. Point cloud was filtered, and large groups of points algorithm were removed based on the Hausdorff distance. The type Cheneau braces were manufactured for patients from 3D models. The STL models were reconstructed and sent to the orthotics manufacturer. The Cheneau type braces were created from the three-dimensional mold as a positive. The step-by-step technique of the brace manufacturing included: initial smoothing, correction in the sagittal plane and pressure application in determined areas. The personalized model for the Cheneau brace was made by the carving robot that carved the model in a 1:1 scale. Finally, the brace was made by the orthotist who put the thermoplastic material over the carved model (Fig. 1). The thermoplastic cooling enabled the pre-prepared form becomes suitable for trial on the real patient.

Purpose Bracing has been reported to be an effective conservative treatment of adolescent idiopathic scoliosis [1]. The brace design must be

123

Int J CARS Conclusion Presented use of a 3D structured light four directional system presents the ability to improve the design and adjustment of scoliotic braces, achieve 3-dimensional correction of scoliotic curves and usefulness for the spinal curvatures and deformity monitoring based on threedimensional visualization of the spinal curves and the external shape of the trunk. The study confirms that 3D model derived from the 3D surface scanning can serve accurately for manufacturing of the orthopedic brace for scoliotic cases. Authors assume that 3D 360 degrees surface scan may significantly improve the accuracy of personalized brace treatment. Acknowledgement This study is supported by the project Nr NR13-0109-10/2011 funded by the National Centre for Research and Development. References [1] Weinstein SL, Dolan LA, Wright JG, Dobbs MB Effects of bracing in adolescents with idiopathic scoliosis. N Engl J Med. 2013; 17;369(16):1512–21. [2] Desbiens-Blais F, Clin J, Parent S, Labelle H, Aubin CE New brace design combining CAD/CAM and biomechanical simulation for the treatment of adolescent idiopathic scoliosis. Clin Biomech (Bristol, Avon). 2012 Dec;27(10):999–1005. [3] Labelle H, Bellefleur C, Joncas J, Aubin CE, Cheriet F Preliminary evaluation of a computer-assisted tool for the design and adjustment of braces in idiopathic scoliosis: a prospective and randomized study. Spine (Phila Pa 1976). 2007 Apr 15;32(8):835–43. [4] Michon´ski J, Glinkowski W, Witkowski M, Sitnik R Automatic recognition of surface landmarks of anatomical structures of back and posture. J Biomed Opt. 2012 May;17(5):056015. [5] Glinkowski W, Michon´ski J, Zukowska A, Glinkowska B, Sitnik R, Go´recki A The time effectiveness of three-dimensional telediagnostic postural screening of back curvatures and scoliosis. Telemed J E Health. 2014 Jan;20(1):11–7.

Augmented reality tool created to hepato-pancreatic surgery on a commercial-available C-arm: good accuracy with laparoscopic ultrasound R. Gobbo Garcia1, M. Reis1, A. Tachibana1, R. Baroni1, B. Lima1, J. L. Siva2, P. Zimmer1, P. Costa1, A. L. Macedo1 1 Hospital Israelita Albert Einstein, Robotic and Hepatobiliary Surgery, Bioengineering and Imaging/Interventional Radiology, Sao Paulo, Brazil 2 Siemens do Brasil, Sao Paulo, Brazil Keywords Augmented reality  Laparoscopic surgery  3D data fusion  Hepato-pancreatic surgery

Fig. 1 The view of the back of the patient treated with the personalized brace manufactured with 3D structured light based scan of the surface topography model

123

Purpose Besides laparoscopic surgery be a common practice among surgeons, the support of intraoperative 3D imaging is a new field. This possibility to look behind the surface of an organ after changes in anatomy during surgery enables the surgeon to gain more information [1]. During laparoscopy the correct detection of tumors beneath the organ surface is frequently challenging due the lack of both visual and tactile identification. In particular, minimally invasive liver and pancreatic interventions are very challenging and demand a lot of experience due to the complex anatomy and limited access in the surgery field [2]. Adding intraoperative ultrasound has further refined surgical judgments and improved clinical results. However a lot of limitation are present during laparoscopic ultrasound reducing its efficacy (e.g. impossibility to reach superior or posterior hepatic lesions, poor distinction of some isoechoic pancreatic tumors) [3].

Int J CARS Augmented reality tools (ART) may be a useful technology to surpass some of those ultrasound drawbacks, improving the planning data in the operation room during the intervention. However such resources are not still commercially available, and most of them not yet approved by the health regulatory authorities worldwide. In this study we show an ART easily created on a commercial available C-arm equipment, adapting some of its built-in softwares to identify oncological targets on two different hepato-pancreatic laparoscopic surgeries: 1—resection of a primary pancreatic neuroendocrine tumor of pancreas and another three liver metastasis; 2— resection of a cystic carcinoma of body of pancreas. Also, we compared the ART findings with the laparoscopic ultrasound images in real time. Methods 3D data fusion of preoperatively contrast-enhanced CT-scan and intraoperative contrast-enhanced cone-bean CT acquired in a commercial available C-arm equipment (Artis Zeego–Siemens) were manually overlayed by two experienced abdominal radiologists, using dedicated workstation and softwares (multimodality fusion Syngo Inspace 3D Siemens and Embolization Guidance Siemens). Manual fusion of those different imaging acquisions were necessary because the significant abdominal anatomic changes after creation of surgical pneumoperitoneum (CO2 inflation into peritoneal cavity) in the laparoscopy [4]. The target-lesions and some others anatomic landmarks (e.g. branches of portal veins e hepatic arteries, main pancreatic duct) were then displayed in a live fluoroscopy image to guide the position of laparoscopic ultrasound probe and others surgical instruments. Results All the targets depicted in real time fluoroscopy had excellent visual correlation with the laparoscopic ultrasound findings (Fig. 1).

Conclusion Using an ART based on commercial available C-arm equipment, we found in real time all the target-lesions previously selected. This performance was visually confirmed by laparoscopic ultrasound. Additionally, for visualization, some anatomical structures or intervention plans were feasible, depending on the surgeons requests. Moreover, based on its excellent visual accuracy, the system is possible applicable for additional intervention schemes such as radiofrequency ablation, percutaneous biopsy or ethanol injection therapy. References [1] Marescaux J, Diana M (2015) Next step in minimally invasive surgery: hybrid image-guided surgery. J. Pediatr. Surg 50: 30–36. [2] Kenngott HG, Wagner M, Gondan M, et al. (2014) Real-time image guidance in laparoscopic liver surgery: first clinical experience with a guidance system based on intraoperative CT imaging. Surg Endosc 28: 933–940. [3] D’Onofrio M, Vecchiato F, Faccioli N, et al. (2007) Ultrasonography of the pancreas—Intraoperative imaging. Abdom Imaging 32: 200–206. [4] Zijlmans M, Langø T, Hofstad EF, et al. (2012) Navigated laparoscopy–liver shift and deformation due to pneumoperitoneum in an animal model. Minim Invasive Ther Allied Technol 21: 241–248.

Using the CustusX toolkit to create bronchoscopy application: Fraxinus

an image

guided

J.B.L. Bakeng1, E.F. Hofstad1, O.V. Solberg1, G.A. Tangen1, T. Amundsen2, T. Langø1, C. Askeland1, I. Reinertsen1, T. Selbekk1, H.O. Leira2 1 Dept. Medical Technology, SINTEF Technology and Society, Trondheim, Norway 2 Dept. of Circulation and Medical Imaging, Faculty of Medicine, Norwegian University of Science and Technology, Trondheim, Norway Keywords Image-guided planning  Open source  Bronchoscopy  Software development

Fig. 1 Augmented reality and intraoperative ultrasound targeting a cystic pancreatic cancer We also tested a series of others applications, simulating tactical percutaneous approaches as used during US-guided ablations and biopsies, all of them with a very good correlation with the transabdominal ultrasound findings (Fig. 2).

Fig. 2 Augmented reality during a laparoscopic liver resection. Clockwise: a- Hybrid Room team; b-Laparoscopic ultrasound probe; c- Carm fluoroscopy; d-Augmented reality tool on fluoroscopy screen

Purpose The aim of this work is to show how a specialized planning and guidance application, Fraxinus, can be built on top of the CustusX platform [1] (www.custusx.org), which is an open source imageguided intervention software platform. We present how the CustusX toolkit is utilized to build a specialized application for a specific clinical procedure and how this can improve the user experience by mirroring the clinical workflow, presenting context sensitive guidance, displaying useful widgets and providing automatic system state transitions. Methods The application specifications were defined through an iterative process involving both technical and clinical personnel. In order to design a system that meets the clinical requirements, it is critically important for technical personnel to understand the tasks and challenges faced by the clinicians before and during the procedure. To gain first-hand experience with the procedure, the software developers attended bronchoscopy procedures. The system specifications were then further refined through numerous discussions where all team members contributed to the process. This resulted in the description of the graphical user interface (GUI), a definition of the workflow and a set of required features.

123

Int J CARS

Specializing CustusX to fit these needs involved: Adding state-of-the-art algorithms for extraction of anatomic structures like vessels, airways and the centerlines of the airways. Hiding unnecessary features from the graphical user interface (GUI). Optimizing the GUI and visualizations to fit the workflow specified. Automating actions that can be predicted based on the procedure. Creating the Fraxinus application. The CustusX toolkit is maintained by a superbuild system [1], which upgrades, configures and builds all components as required, like external libraries or plugins. Fraxinus was added as another layer on top of the CustusX superbuild system. New functionality is easily added as plugin building blocks in the image-guided framework. Results Fraxinus has been developed and will be released as an open source planning and guidance application built on top of CustusX. The user interface of CustusX has been modified to enhance information, quality assurance and user friendliness with the intention to increase the overall yield for the patient. The workflow in the Fraxinus application includes: importing DICOM images, airways and centerline extraction automatically by using the algorithm described in [2], pin-pointing a target position (for biopsy or aspiration) and generation of a route-to-target along the airway centerline. Finally a virtual bronchoscopy (VB) is created with a virtual travel through the lung tree from the top of the trachea, through the bronchi, to the target. Two modes of VB exists: a fly-through mode similar to the video from a bronchoscope (Fig. 1) and a novel VB method creating CT cut-planes defined by the current virtual position and the airway orientation (Fig. 2). The user controls the position, view direction and rotation of the virtual bronchoscope (Fig. 1). As the workflow of the procedure is relatively constant, some actions are predicted and automatically performed by the application.

Fig. 1 Fraxinus in VB fly-through mode from a position along the route-to-target inside the airways of a phantom. Position, view direction and rotation are controlled at the bottom of the screen

123

Fig. 2 The VB cut-plane mode in Fraxinus Conclusion Fraxinus is a specialized image-guided planning application for bronchoscopy built on top of the open source image-guided intervention software platform, CustusX. The CustusX toolkit facilitated the development by offering extension points into many of the important steps of the software development process. Addressing the clinical needs in bronchoscopy have resulted in a tool that can offer free assistance to clinicians performing these procedures, currently within a research environment. The application is an example of an open source solution targeting a specific clinical need, and that is freely available for further development and research. The software has the potential to become a useful tool for the future patient treatment. References [1] Askeland C, Solberg O, Bakeng J, Reinertsen I, Tangen G, Hofstad E, Iversen D, Va˚penstad C, Selbekk, Langø T, Hernes T, Olav Leira H, Unsga˚rd G, Lindseth F (2015) Custusx: an open-source research platform for image-guided therapy. International Journal of Computer Assisted Radiology and Surgery, pages 1–15, 2015. ISSN 1861-6410. doi:10.1007/s11548-015-1292-0. [2] Smistad E, Elster AC, Lindseth F (2014) GPU accelerated segmentation and centerline extraction of tubular structures from medical images. Int J Comput Assist Radiol Surg, 9(4):561–75, 2014. doi:10.1007/s11548-013-0956-x.

Int J CARS Clinical data analysis of surgeon’s arm supporting device ‘iArmS’ 1

1

1

2

T. Goto , Y. Hara , K. Hongo , H. Okuda , M. Fujimura 1 Shinshu university, Department of Neurosurgery School of Medicine, Matsumoto, Japan 2 Denso corporation, Kariya, Japan

2

Keywords Microscopic neurosurgery  Surgeon supporting device  Robotic surgery  Armrest Purpose Neurosurgeons must perform fine microscopic neurosurgical procedures in a long time and unstable condition of surgeon’s arm. Long time surgery causes surgeon’s fatigue, it may decrease surgical quality. Unstable condition of surgeon’s arm causes arm tremble, it may also decrease surgical quality. To dissolve these problems, an intelligent arm supporter, iArmS, has been developed for assisting microscopic neurosurgery [1, 2, 3], and was put in the market since the spring of 2015. The iArmS moves by following surgeon’s arm and holds surgeon’s arm anywhere. The iArmS consists of an arm holder and an robot arm. The re-usable arm holder is made from the carbon. The robot arm has 5 degrees of freedom. Each joint has counter-weights, electric brakes, and encoders but no electric motors. A force sensor is set between the arm holder and the holder support. The brakes are controlled by judging the information from force sensor and encoders (Fig. 1). The action of the iArmS is classified in three modes: wait, hold, amd free. Wait: the arm holder keeps the position when surgeon’s arm is away from the arm holder, hold: the arm holder keeps the position and holds the surgeon’s arm, free: the arm holder moves by following surgeon’s arm. The program to change the each mode is called as ‘‘state’’. In static state, the iArmS is basically in the hold mode, and in dynamic state, the iArmS is basically in the free mode. The surgeon can select one from two states by his/her choice. The system is attached to the operator’s chair (Micro Chair, Mizuho Ikakogyo Co., Ltd., Tokyo, Japan) and follows with the chair. In this report, we analyzed the logging data of the iArmS.

total 56 surgeries. 17 pituitary adenomas, 8 meningiomas, 6 vestibular schwannomas and 6 gliomas were removed in tumor surgery. Six clippings for aneurysm, four carotid endoarterectomies for carotid artery stenosis, one microvascular decompression for hemifacial spasm were conducted. The data of force sensor and mode were related with the process of surgery. Results Case 1: Clipping surgery was undertaken for an gradually enlarging unruptured right internal carotid artery aneurysm in a 61-year-old woman. After applying the routine right frontotemporal craniotomy, clipping procedure was performed via a trans-sylvian approach. A left-handed surgeon sit on the microchair and two sets of iArmS were used under both arms throughout the microscopic procedure. The data of force sensor and the mode were averaged in each one second and analyzed. Time of hold mode was 6002 s. (63 %) in the dominant hand and 7952 s. (83 %) in the non-dominant hand in the total 9543 s. of microscopic procedure. The mean force was 16 N in the dominant hand and 28 N in the non-dominant hand in the hold mode. The process of the clipping surgery was divided in five segments: distal sylvian dissection, medial sylvian dissection, optic nerve dissection, aneurysm dissection and clipping. Hold mode ratio in each process gradually decreased in both hands. The force in the dominant hand gradually decreased in surgical proceeding, but it was regular in the non-dominant hand (Fig. 2)

Fig. 2 Graph of force sensor in the case 1. The force in the dominant hand gradually decreased in surgical proceeding, but it was regular in the non-dominant hand

Fig. 1 Photograph of the iArmS. iArmS is used in microscopic neurosurgery. iArmS supports surgeon’s arm. Surgeon sits on the operator’s chair and the iArmS follows with the chair Methods The iArmS was used in total 56 surgeries from February to August 2015 by mainly 6 board neurosurgeons. The iArmS sampled and analyzed the data of force sensor and the mode in each 20 mS. The clinical data of iArmS were stored. The tendency of usage of the iArmS was analyzed. There were 45 tumors and 11 vascular lesions in

Case 2: Unruptured gradually enlarged right middle cerebral artery aneurysm was clipped in a 63-year-old man. Clipping procedure and use of the iArmS were as the same condition as in case 1. 2,526 s of distal sylvian fissure dissection was performed by young neurosurgeon using the iArmS in the dominant hand. Frequency of free mode was only one time in his procedure. It was less frequent than that of a board neurosurgeon. Conclusion By analyzing the data of iArmS, the process of surgical procedures can be analyzed among surgeons. This may contribute to education for young neurosurgeons to establish smooth and efficient surgery. References [1] Goto T, Hongo K, Yako T, Hara Y, Okamoto J, Toyoda K, Fujie MG, Iseki H. The Concept and Feasibility of EXPERT:

123

Int J CARS

[2]

[3]

Intelligent Armrest Using Robotics Technology. Neurosurgery. 72 Suppl 1:39–42. 2013. Hara Y, Goto T, Okamoto J, Okuda H, Iseki H, Hongo K. An Armrest is Effective for Reducing Hand Tremble in Neurosurgeons. Neurol Med Chir (Tokyo). 55:311–6, 2015. Yako T, Goto T, Hongo K. Usefulness and limitation of a freely movable armrest in Microneurosurgery. International Journal of Neurology and Neurosurgery 1 (4) 185–190, 2009.

Development of forceps with continuous suction function for resecting brain tumors F. Shimizu1, A. Hanafusa1, K. Masamune2, Y. Muragaki2, H. Iseki3 1 Shibaura Institute of Technology, Saitama, Japan 2 Tokyo Women’s Medical University, Tokyo, Japan 3 Waseda University, Tokyo, Japan Keywords Neurosurgery  Brain tumors  Forceps  Suction Purpose Doctors use small pairs of forceps to resect brain tumors. The shape of the forceps resembles a pair of scissors, and there is a small cup on the tip of the forceps to resect tumors. Since the cup and the resecting tumor volume are both small, the forceps must be frequently inserted into and removed from the operative field in order to resect the entire tumor [1]. This process requires time and effort on the part of the doctor. One possible solution to the problem is to use an ultrasonic aspiration surgical tool. This tool destroys tumors by means of ultrasonic waves, and is able to aspirate tumors continuously with normal saline solution. However, this method requires a wide operative field to prevent the propagation of ultrasonic vibrations to other tissues. Moreover, since this method destroys the tumor, it precludes histopathological inspections. The purpose of this study is to develop forceps that can resect tumors continuously without destroying the tumor texture, in order to facilitate an effective and efficient operation for the doctor. Methods The system configuration consists of the newly developed forceps, a vacuum pump, a regulator, a pressure sensor, a sensor that detects tip opening and closing, and a computer that controls the system (Fig. 1). The tip of the forceps is closed upon insertion to the operative field, and opened in the proximity field of the tumors. The tumor is drawn into the tip by the suction system, and is resected by closing the tip. Finally, the pieces of resected tumor are collected in a trap within the suction system. Because the tip is closed when the tumor is suctioned, the process is safe and causes no damage to surrounding tissue. The suction process and air pressure are controlled using the sensors and regulator connected to the computer.

Fig. 1 System configuration

123

The key component of the system is the forceps, which have a continuous suction function. The specification of the forceps was determined through observation of surgical operations and discussion with doctors, and the mechanism and structure were designed accordingly. The developed forceps are shown in Fig. 2. The trial forceps are composed of a double pipe structure with an outer diameter of 3 [mm]. An inner pipe is used to transport tumor pieces during aspiration, and an outer pipe is used to open and close the tip of the forceps by sliding along the inner pipe. The tip, which resects tumors, is connected to the inner pipe by spring plates that are bent outward. Therefore, the tip opens when the outer pipe slides backward, and closes when the spring plate is covered by sliding the outer pipe forward. In addition, a pair of bayonet-type tweezers is connected to the outer pipe. When the tweezers are nipped and pushed, the outer pipe slides forward, closing the tip; when the tweezers are released and opened, the outer pipe slides backward, opening the tip. Trial forceps were manufactured with stainless steel, and tip-opening and tip-closing tests, as well as suction tests, were performed using a tumor model.

Fig. 2 Developed trial forceps when the tip is open Results The tip of the forceps was opened and closed by sliding the outer pipe, and the maximum opening angle was 76 []. Suction experiments were performed using tofu as an alternative tumor model. Two different outer pipes having inner diameters of 2.75 and 2.85 [mm], respectively, were used in the experiment. Since the external diameter of the inner pipe was 2.7 [mm], the clearances were 0.025 and 0.075 [mm], and the pipes were termed small clearance pipe and large clearance pipe, respectively. After the tumor model was resected by closing the tip, it was suctioned by setting the regulator pressure value to -50 [kPa]. The time variation of the air pressure was measured by the pressure sensor during suction. In the case of the large clearance pipe, the pressure continued decreasing until it reached -28 [kPa], and in the case of the small clearance pipe, the pressure decreased until it reached -50 [kPa], equal to a set point of the regulator. The tumor model was successfully suctioned in all five trials using the large clearance pipe. However, when the small clearance pipe was used, the tumor model was successfully suctioned in only three out of five trials. These results show that when the small clearance pipe was used, airflow was obstructed, compromising the movement of the tumor model. However, in the case of the large clearance pipe, the fact that the air pressure did not drop all the way to the regulator set value indicated the ability of air to flow through the tips that were not entirely closed or through the gap between the tip and the outer pipe. This identifies airflow as an element necessary for effective suction and transportation of the resected tumor model. Conclusion The mechanism and structure of the forceps having an outer diameter of 3 [mm] were designed and manufactured. The tip could be opened and closed by closing and releasing the bayonet-type tweezers connected to the sliding outer pipe. Suction experiments were performed using the developed forceps to resect alternative tumor models. Two different outer pipes, one

Int J CARS having a small and the other having a large clearance between the inner and outer pipe, were tested. The tumor model could effectively be suctioned and transported in all trials using the large clearance pipe, in which airflow was present. Future studies include evaluation of the strength and durability of trial forceps, improvement of the mechanism, and development of a total suction system. References [1] Suzuki T, Ishihara S (2010) Neuroendoscopic surgery for brain tumors. Japanese Journal of Clinical Medicine (68):368–374[in Japanese].

One-stage skull tumor excision and reconstruction surgery with pre-fabricated 3D rapid-prototyping guide and implant: a clinical study

guidance of resection of the tumor, and the other is a permanent implant used for repairing the skull bone defect. The overall system sketch is illustrated in Fig. 1. One day before the operation, a serial CT data set of the patient’s head was acquired. It was transferred into a computer system containing an image visualization software. A three-dimensional image model of the skull with the bone lesion was reconstructed first. The image model was then managed by an animation software. To design the excision area, the HF value of the lesion are measured on CT. A threshold of 100 HF value are used for primary delinearation of the lesion then a 0.5–1 cm safe margin was created. Real-sized resin biomodels of the skull base and the implant were fabricated by a stereolithographic device (Prodigy Plus, Stratasys, Inc., Eden Prairie, Minn., USA). The stereolithographic models were extremely accurate and served as preoperative simulators.

C.-T. Wu1, S.-T. Lee1 1 Chang-Gung Memorial Hospital, Neurosurgery, Kweishan, Taiwan, Republic Of China Keywords Rapid prototyping  Skull bone tumor surgery  Surgical simulation  Computer aided surgery Purpose Computer-aided design (CAD) and rapid prototyping technique has gained increasing popularity in clinical practice. But in real surgery, the product of CAD system needed to be placed to the target precisely and anatomically [1, 2]. The objective of our study was to propose a novel surgical guidance method that incorporates CAD bone surface masks that serves as the positioning reference, for skull bone tumor excision and bone defect reconstruction. There are several kinds of skull bone tumor need surgical excision and reconstruction at the same time. The lesions can be locally aggressive, so definitive surgical excision typically is recommended when possible. Traditionally, excision of the lesion and repair of the defect were performed by direct vision of surgeons and judged by their experience [3, 4]. Complete surgical excision of lesion proved to be beneficial for less recurrence rate and avoid adjuvant chemotherapy. In our previous study and works, we have proved that the custom prefabricated CAD/CAM polymethyl methacrylate (PMMA) prostheses for repairing skull defect are effective. Cranioplasty using the patient specific implant has several advantages including reduced surgical time, surgical blood loss, technical simplicity, and satisfactory aesthetic result [2, 5]. In this paper, we propose to use a dedicated patient specific and CAD/CAM surgical guide system for simultaneously preoperative surgical jig and implant design and production. Our CAD/CAM system is composed of two subsystems: including a virtual surgical planning and a 3D-printing sub-system. The output products of the system are 2 patient-specific bio-implants that can be made pre-operatively. The 2 bio-models, one is a surgical jig used for guidance of excision of the tumor, and the other is a permanent implant used for repairing the skull bone defect. Our system enables simultaneous resection and repairing through limited incisions, so as to benefit the surgeons with reduced surgical complexity, and to benefit the patients with reduced surgical invasiveness and potentially reduced operation duration, hence to minimize the incidence of complications. We validated our system qualitatively with six patient cases to show the feasibility of applying this system clinically. The reliability of the system is also assessed based on this pilot study group. Methods The system is composed of two subsystems: including a virtual surgical planning subsystem and a 3D-printing subsystem. The output products of the system are 2 patient-specific bio-masks that can be made pre-operatively. The 2 bio-models, one is a surgical jig used for

Fig. 1 The overall system is composed of a virtual surgical planning subsystem and a 3D-printing subsystem. Based on input CT data, a virtual model with the original lesion is reconstructed and visualized. This virtual model 1 is then managed to design a resection area and create model 2 with a skull defect. Model 2 is further managed by animation and creates a fully reconstructed model 3. According to the designing, two output models will be created: 1a resection model, it is used for a surgical jig guiding surgeon to resect bone tumor at operation; 2a reconstruction mode, for manufactueing the prefabricated cranioplasty PMMA flap Results The implants designed with the proposed system were successfully applied to all cases through minimally innvasive surgeries. After the treatments, no further complications were reported. No problems were encountered during the entire procedure from skull tumor resection to reconstruction. Owing to the high precision of the patient-specific resection and fixation guides, we were able to remove tumor and place the patient-specific implant in a safe, rapid and targeted manner The proposed planning software system was successfully used to plan 7 skull tumor excision and repair procedures. The intra-operative observation find adequate fitness of resection guide to the lesion, and contour of surrounding outer surface. Then bone cutting procedure were smoothly performed. The implantation of prefabricated bone substitute flap and fixation process were easy . We use post-operative CT scan for quantitively validation. The contour of outer surface through coronal and sagittal reconstruction image are smooth. Conclusion Computer-assisted design and manufacturing of patient-specific CAD/CAM-fabricated surgical guide and implant is a promising approach for surgery of skull lesions. The likely benefits of this method include a reduction in operating time and an improvement in the quality of reconstructions especially in terms of cosmesis, extent of surgical margin and function. Clinical trials are required in order to determine whether these promising results can be applied to more complicated and complex bony area like skull base and craniofacial and what further developments are needed in the future.

123

Int J CARS References [1] Yu H, Wang X, Zhang S, Zhang L, Xin P, Shen SG (2013) Navigation-guided en bloc resection and defect reconstruction of craniomaxillary bony tumours. International journal of oral and maxillofacial surgery, 42(11), 1409–1413. [2] Wu CT, Lee ST, Chen JF, Lin KL, Yen SH (2008) Computeraided design for three-dimensional titanium mesh used for repairing skull base bone defect in pediatric neurofibromatosis type 1. Pediatric neurosurgery, 44(2), 133–139. [3] Rengier F, Mehndiratta A, von Tengg-Kobligk H, Zechmann CM, Unterhinninghofen R, Kauczor HU, Giesel FL (2010) 3D printing based on imaging data: review of medical applications. International journal of computer assisted radiology and surgery, 5(4), 335–341. [4] Gellrich NC, Barth EL, Zizelmann C, Ru¨ker M, Scho¨n R, Schramm A (2006) Computer assisted oral and maxillofacial reconstruction. CIT. Journal of computing and information technology, 14(1), 71–77. [5] Lee SC, Wu CT, Lee ST, Chen PJ (2009) Cranioplasty using polymethyl methacrylate prostheses. Journal of clinical neuroscience, 16(1), 56–63.

registered onto the 3D model of these optical scans. The guiding surfaces were marked and matching target cut planes were defined. Consequently, this definition of the target planes eliminated the potential error of the accuracy of the fit of the guide onto the synthetic bone. Once the different osteotomies were performed by all surgeons, the resected fragments were optically scanned and registered on the 3D models used to place the cuts in 3-Matic. The resected surfaces were marked and best fitting planes were calculated.

Do patient-specific cutting guides enhance the accuracy of a single diaphyseal osteotomy? G. Sys1, G. Lenaerts2, K. Chellaoui2, F. Shumelinsky3, C. Robbrecht1, B. Poffyn1 1 Ghent University Hospital, Department of Physical medicine and orthopaedic surgery, Ghent, Belgium 2 Materialise, Leuven, Belgium 3 Universite´ Libre de Bruxelles, Department of Orthopedic Surgery, Institut Jules Bordet, Brussels, Belgium Keywords Osteotomy  Patient-specific  Cutting guides  Accuracy Purpose Osteotomies for bone tumor resections require the highest surgical precision in order to achieve adequate tumor margins. This study explores whether the use of patient-specific cutting guides for a single, diaphyseal osteotomy on a synthetic femoral bone results in a more accurate osteotomy in the three anatomical planes when compared to a freehand technique, using 3D virtual models based on optical scans. Differences in guide design were analysed by comparing open and slot guides, the effect of oscillating and reciprocating saw blades and surgical experience. Methods Two experienced oncological orthopaedic surgeons and one intraining resident performed each three types of osteotomies: (1) with open guides; (2) with slot guides; (3) freehand. The osteotomies with guides were done with an oscillating and a reciprocating saw blade. The freehand osteotomies were performed with a reciprocating saw blade as preferred by the surgeons. Each cut was performed three times in a proximal, medial and distal part of the diaphysis of a synthetic femoral bone. For the planning of the osteotomies, the synthetic bones were optically scanned and converted into virtual 3D models and a reference coordinate system was created (3-Matic, Materialise NV, Leuven, Belgium). Target cut planes were defined for the different osteotomies and open and slot guides were designed for the corresponding surgical plan. For the freehand osteotomy, target cut planes were defined parallel to the axial planes of the femoral bone and intersecting with marked entry spots (Fig. 1). For the guided osteotomies, open and slot guides were designed and manufactured in polyamide using additive manufacturing technology. Then, the synthetic bones were optically scanned with the affixed guides. Virtual models of the guides were

123

Fig. 1 Distance between the target and the resection plane is calculated as the average of the distances between the target and the cut planes (d1 and d2) on their intersection with the longitudinal axis The difference between the target and the performed resections was calculated as the average difference in distance (ADD) between target and the cut planes on each side of the blade at the intersection of the planes with the longitudinal axis (Fig. 1). Average angular deviations were quantified by measuring the intersection angle between the target plane and cut plane in the sagittal (AADS) and coronal (AADC) planes. For each type of osteotomy, mean deviations from the target cut were calculated. The Kruskal–Wallis test was used to analyse the differences in accuracy between the, different osteotomy techniques, saw blades and operators (SPSS 22.0, IBM, NY, USA; p \ 0.05). Results Overall, ADD ranged from 0.01 to 2.09 mm, AADS ranged from 0.0 to 2.75 and AADC ranged from 0.02 to 4.14. When comparing the accuracy of the open guided versus slot guided and freehand osteotomies, averaged for all surgeons and both sawblade types, the deviation from the target cut was significantly larger for the freehand osteotomy compared to both guided osteotomy for ADD (p = 0.025), AADS (p = 0.022) and AADC (p = 0.014). There were no significant inter-operator differences for the freehand osteotomy. When comparing both guides and both saw types, no significant inter-operator differences were found. When analysing the effect of saw type, the use of a reciprocating saw resulted in a significant difference between osteotomies performed with guides compared to freehand osteotomies. The freehand osteotomies showed significantly larger deviations for ADD (p = 0.027) and AADS (p = 0.001) and larger, but not significant, deviations for AADC. The use of an oscillating saw resulted in significantly larger AADC (p = 0.019) with the open guide compared to the slot guide. When analysing the effect of guide type, the use of slot guides did not lead to significant differences in accuracy for saw type or for surgeons. However, larger deviations were found for the oscillating saw compared to the reciprocating saw. When open guides were used, a significant difference was found according to saw type for AADS (p = 0.047) and AADC (p = 0.015): a markedly larger distribution for the oscillating saw (AADS range -1.25 to 0.39; AADC range

Int J CARS -4.94 to 1.59) was observed when compared to the reciprocating saw (AADS range -0.82 to 0.48; AADC range -2.79 to 0.78). Again, no significant inter-operator differences were found for accuracy of osteotomies with open or slot guides. Conclusion The use of patient-specific cutting guides for a single diaphyseal osteotomy resulted in a more accurate cut in all planes compared to a freehand osteotomy. Reciprocating saw blades and slot guides lead to a more accurate cut in all planes compared to oscillating saw blades and open guides. Surgical experience did not influence the accuracy of the cut. Therefore, patient-specific cutting guides are a valid tool for enhancing the accuracy in osteotomy procedures.

Evaluation of a new IGS for the implantation of proximal femoral interlocking nails F. Sommer1, S. Grote2, Ch. Becker1, A. Greiner1, B. Rubenbauer1, Ch. Linhart1, S. Weidert1 1 Ludwig-Maximilians-Universita¨t, Department of Traumatology, Munich, Germany 2 Klinikum St. Elisabeth, Department of Traumatology, Straubing, Germany Keywords Image guided surgery  Gamma nail  Intertrochanteric fracture  Stryker adapt  Randomized clinical trial Purpose Intertrochanteric fractures are a common injury especially in the elderly. Due to epidemiological changes in our society, we will have an increasing number of elder patients in future, and the incidence of intertrochanteric fractures will increase tremendously over the next decades (1). The main problem of intertrochanteric fractures is the instability and further dislocation of the fracture with weight bearing. Fracture reduction on an extension table and osteosynthesis is therefore the most common treatment strategy. Closed reduction and intramedullary nail osteosynthesis with implants having a lag screw for the femoral head are widely used (2). To be effective, the precise placement and lenght of the lag screw is crucial as the calculated Tip-Apex-Distance (TAD) should not exceed 25 mm (3). Commonly, this procedure is done with the help of fluoroscopy and implant position and lenght of the screw are measured indirectly with tools. This study aims at determining whether the use of a navigation system (Stryker ADAPT) that analyses 3D implant position on the fluoro image by the help of a radiodense clip on the aiming device and provides help in measuring and aiming for the optimal lag screw position. The study is ongoing and we want to present the intermediate results. A total of 17 patients was included so far in this study. Methods Inclusion criteria was fracture classification (31 A1 and 31 A2 fracture types according to AO classification), eligibility for a 180 mm nail and patient consent. Participants were randomized into two groups (n = 13 navigated vs. n = 5 control group) with the patient blinded to the result and the surgeon being told right in the operating room. In the control group, the navigation screen was not shown to the surgeon and the ADAPT clip was not utilized. All fluoro images were automatically stored on the system and then evaluated. Main outcome criterion was TAD, secondary outcome parameters were operation time and drilling attempts of the lag screw’s guide wire. Results Results showed a significantly lower TAD (mean 20,97 mm vs. 29,36 mm with the goal of aiming at \25 mm; p = 0,019; a = 0,05; CI 95 %) in the navigation group as well as significant less drilling attempts (mean 1,67 vs. 4,00 attempts; p = 0,048; a = 0,05; CI 95 %) and even less, but not statistically significant, operation time so

far (mean 35 min vs. 49 min; p = 0,064; a = 0,05; CI 95 %). Our preliminary results show that the system seems more efficient in helping to place the lag screw within the recommended limits than normal fluoroscopy. Furthermore, the multiple drilling attempts to determine the right angle in placing the guide-wire of the lag screw in fluoroscopy. Conclusion Our preliminary results show that the system helps the surgeon to place the lag screw more precisely. Unlike common navigation systems the ADAPT does rarely need user interaction and thus does not add any additional time to the surgery. In contrary it even seems to save time which seems mainly attributable to the fact that less drilling attempt were needed. Additional less drilling attempts reduce unneeded harm to the patient’s bone. We expect our results to become even more clear during the ongoing study. The ADAPT system seems to offer additional precision and more linear workflow at no additional operation time and without the need of user interaction References [1] Bonnaire F, Straßberger C, Kieb M, Bula P (2012) Osteoporotic fractures of the proximal femur. What’s new? Der Chirurg; Zeitschrift fur alle Gebiete der operativen Medizin. 83.10: 882–891. [2] Baumgaertner MR (2003) Intertrochanteric hip fractures. Jupiter JB, Levine AM, Trafton PG. (Eds.) Brownder BD. Skeletal Trauma: Basic Science, Management, and Reconstruction. Philadelphia: Elsevier. [3] Baumgaertner MR, Curtin SL, Lindskog DM, Keggi JM (1995) The value of the tip-apex distance in predicting failure of fixation of peritrochanteric fractures of the hip. J Bone Joint Surg Am., 77:1058–1064.

The three dimensional model for the microtia plasty: the new design microtiaplasty with the tissue expansion Y. Takeichi1,2, H. Iguchi3, H. Motai4, H. Tada2, S. Itho5 1 Daiyukai Daiichi Hospital, Plastic and Reconstrucive Surgery, Nagoya, Japan 2 Wakaba Hospital, Plastic and Reconstrucive Surgery, Thu, Japan 3 Nagoya City University, Arthroplastic Medicine, Nagoya, Japan 4 Motai Clinic, ORL, Tokai, Japan 5 Aichi Medical University, Plastic and Reconstrucive Surgery, Nagakute, Japan Keywords Mirror image  3D model  Microtia  Tissue expander Purpose The form of an ear has 3 dimensional complex structures. Therefore, in microtiaplasty, a plastic surgeon requires a 3 dimensional normal ear model as with a sculpture requires a human model. We have done the microtia reconstructions using tissue expanders to cover such 3 dimensional complex flames. To fit the reconstructed ear to the opposite normal ear, we made the mirror image model of the normal ear. Methods For the microtiaplasty, we make a three dimensional ear model of the mirror images of a opposite normal ear. CAT scan DICOM data of the opposite ears are transferred to STL format CAD data by Mimics (Materialize, Belgium). The mirror images of normal ears were machined by a desktop computer numeric machine (Modella MDX40A, Roland DG, Japan) or a 3D printer (Replicator 2X, MarkerBot, USA). We do microtiaplasty by a costal cartilage 3D frame and the tissue expansion. We make an ear framework by 6th, 7th, 8th costal cartilage. This framework is very thick, therefore we use the tissue expansion to cover. We have used PMT’s 5 9 6 cm double chambered tissue expander, and expanded to 200 cc.

123

Int J CARS Results We have made 35 pieces of 3D ear models for the microtia reconstructions (Fig. 1). In all cases, we could reconstruct precise forms, especially depth of auricular concha and antihelix. The most of reconstructed ears are very similar to the opposite normal ones (Fig. 2). All patients have been satisfied with the results.

Fig. 1 The 3D model of the mirror image of opposite normal ear (left) and the 3D framework made by costal cartilage (right)

model for the microtia reconstruction is just like a real human model for a sculpture. We have made 3D frameworks these were faithfully reflected to the 3D models. These frameworks were built with threedimensional design, hence they were very thick. To cover these thick frameworks, the large area of the skin should be required. We have used PMT’s 5 9 6 cm double chambered tissue expander, and expanded to 200 cc. We elevate two triangle flap on the tissue expanded region. The frontal side of the reconstructed ear is placed the frontal triangle flap and the back side of the ear is covered by the posterior triangle flap. The temporal region of the donor site is closed directly to prevent shrinking of the expanded skin. We remove the capsule of the expanded skin to clear the ear contour. But we remain the capsule of the triangle flap region to maintain the blood supply. Most severe problem of this method is exposure of the tissue expander. In our cases, the exposure rate is about less than 5 %. To prevent this complication, we keep the tissue expander with no expansion for one month. After that, we begin to inject saline 2 or 3 ml at one time. We gradually increase the injection volume. The maximum injection volume is 12 ml at one time. The increasing volume curve is just like a sigmoid curve. References [1] Takeichi Y, Igichi H, Motaiet H, et al. (2012) Usage of 3 dimensional preoperative planning and 3 dimensional individualized cutting device for maxillofacial osteotomy, Int J CARS, vol. 7, S418–s418. [2] Takeichi Y, Motai H, Igichi H (2013) Hydroxyapatite block custom made by CNC machine with 3-D CAD can achieve accurate reconstruction of facial deformity with very small bone defect, Int J CARS, vol. 8, S368–s369 [3] Hata Y, Umeda T (2000) Reconstruction of congenital microtia by using a tissue expander. J Med Dent Sci 47: 105–116

Towards a real-time mapping of the pre-operative images in the intra-operative navigation system Y. Luo1, L. Shi2,3, J. Wu4, V. Mok2,3, W. Chu1, D. Wang1,5 1 The Chinese University of Hong Kong, Dept. of Imaging and Interventional Radiology, Hong Kong, Hong Kong 2 The Chinese University of Hong Kong, Department of Medicine and Therapeutics, Hong Kong, Hong Kong 3 The Chinese University of Hong Kong, Chow Yuk Ho Technology Center for Innovative Medicine, Hong Kong, Hong Kong 4 Huashan Hospital, Neurological Surgery Department, Shanghai, China 5 CUHK Shenzhen research institute, Shenzhen, China Keywords Registration  Pre-operative MRI  Intra-operative MRI  Tumor resection

Fig. 2 Pre-operative view (above left). The temporal skin was expanded using a tissue expander. Post-operative view (above right). The 3D costal framework was inserted. Reconstructed ear view (below left) and opposite normal ear view (below right). Both ears look like very similar Conclusion The auricular plasty is a very complex reconstruction. Therefore we could not get enough date from the 2 dimensional template. The 3 D

123

Purpose In tumor surgery, one critical objective is to maximize tumor resection while sparing important areas of the brain and minimizing the risk of inducing permanent neurological deficits. To achieve this goal, the pre-operative MRI and intra-operative MRI are acquired and utilized to guide the tumor resection surgery. In order to fuse the preoperative and intra-operative information, image registration is an essential procedure. Through the registration between pre-operative and intra-operative MRIs, the resulting transformation map can be utilized to map the pre-operative multi-modal images and the planned tumor resection regions to the intra-operative image space, so that the

Int J CARS

multi-modal pre-operative information can serve as the guidance in tumor resection surgery. The critical issue in this procedure is that the image registration process should be fast and accurate. Although the affine registration used in most of the current navigation systems is fast, the degree of freedom (DOF) of the estimated transformation is limited. Non-rigid registration that allows more DOF is preferable in this situation, but the heavy computational cost prohibits its application in the neurosurgery. To address this problem, we developed a fast non-rigid image registration to realize the ‘‘real-time’’ multi-modal image fusion, which can help the neurosurgeon make quick responses during the surgery. Methods We designed a down-sampling scheme based registration acceleration framework and applied it on a GPU-based registration method. The general principle of the acceleration framework is that we first divided the original images into several lower-resolution image partitions through an interleaving down-sampling scheme. The registrations were performed between the corresponding lower-resolution image pairs. Due to the lowered image resolution, the computation time of the registration is reduced. Meanwhile, the registrations of the downsampled image pairs can be executed simultaneously. The whole registration process was thus accelerated in this manner. The final registration result was retrieved through the fusion of those lower-resolution image registration results in an average manner. We applied this scheme on the GPU-based implementation of the SyN registration method. The SyN method was chosen due to its high registration accuracy. However, the high computational cost of the SyN method makes it impractical to be directly applied. We parallelized the similarity metric calculation in the SyN method on a GPUplatform, which consequently speeds up the original registration process. To further improve the efficiency, we applied our downsampling acceleration scheme on the GPU-based SyN method and achieved a great registration runtime reduction. Results The MRI data used in this experiment was collected from five patients who underwent glioma resection surgery. Pre-operative MRIs were acquired in the diagnostic room of an intra-operative MRI-integrated neurosurgical suite equipped with a movable 3.0 T scanner on the day before operation. A T2-FLAIR sequence for glioma without contrastenhancement (TR, 9000 ms; TE, 96 ms; TI, 2500 ms; flip angle, 150; slice thickness, 2 mm; FOV, 240 9 240 mm2; matrix size, 256 9 160) was acquired. All patients received tumor resections in the operating room. An eight-channel opened head coils, which was designed dedicated for the intra-operative scan, was placed onto the patient’s head after the patient was wrapped. The intra-operative sequence was the same as the pre-operative sequence. Apart from the structural MRI, DTI was also acquired by means of a twice-refocused SSSE EPI sequence (TR/TE 7800/92 ms; FOV 230 9 230 mm2 and 128 9 128 matrix, 40 sections with resolution of 1.8 9 1.8 9 3 mm3 with no intersection gap; GRAPPA parallel acquisition technique with a factor 2 to reduce susceptibility artifacts with diffusion-sensitizing gradients along 20 directions by using b-values of 0 and 1000 s/mm2). The registration results of the five clinical cases are shown in Fig. 1.

Fig. 1 The pre-operative and intra-operative image registration results. Each row shows the registration result of one subject. The first column shows the pre-operative MRIs overlaid with the planned tumor resection mask. The second column shows the intra-operative MRIs overlaid with the registered tumor resection mask. The third column shows the warped pre-operative MRIs. The fourth column shows the warped pre-operative FA maps calculated using registered DTIs After registration, both the planned tumor resection regions and the pre-operative DTI images were transformed in the intra-operative MRI space. The registered tumor resection mask and DTI images can be overlaid on the intra-operative MRI as a guidance of the surgery. The original SyN method took an average of 20 ± 2 min. With our accelerated GPU-based SyN method, the average runtime of the 5 pairs of registration was reduced to 2 ± 0.5 min, which saved up to 90 % of the original runtime. Conclusion In this work, we have designed a fast non-rigid image registration to be used in the intra-operative navigation system for assisting neurosurgery. The greatly improved computational efficiency and high registration accuracy push forward the application of non-rigid registration in many ‘‘real-time’’ applications.

123

Int J CARS Fast refined 3D calibration of a Nintendo Wii remote stereo camera rig F. Schoovaerts1, R. Kamouni1, O. De Witte1 1 ULB Erasme Hospital, Neurosurgery, Brussels, Belgium Keywords Calibration  Navigation  Camera  Tracking Purpose The concepts of 3D tracking were first developed for use in missile guidance systems then for visual effects in movies. It has since spread to the game industry and to more and more every day life applications. The virtualisation and augmented reality has entered our pockets as embedded in our smart phones and has many potential utilization within the medical world. Diagnostic and treatment apparatus are already taking advantage of these technologies in fields like radiotherapy, neurosurgery, neurosciences, orthodontics, for patient localisation, transcranial magnetic stimulation, target tracking. A simple and cheap 3D localisation system offers many purposes in modern operation theatre like for example as a mean of interaction with machines when in ‘‘sterile’’ mode. In this paper we propose a method for a fast and refined calibration of a 3D camera rig using a couple of standard game controllers. The popular Zhang [1] calibration algorithm requires a planar grid pattern shown at a few different orientations. The Nintendo Wii remote can follow up to four points concurrently in an arbitrary order. However, four markers seem quite few for a fast and precise grid based calibration. The technique proposed here, while taking advantages of the pros get around the limitations of the hardware. The Camera Calibration Toolbox for Octave/ Matlab [2] has been the central core of the method. Methods The low cost Nintendo Wii remote controller has a high performance infrared camera offering a quite effective resolution at a frequency as high as 100 Hz wirelessly and up to 200 Hz using I2C protocol when wired. In this approach, both the sensors are kept fixed in a specific geometry and infrared sources or reflectors are manipulated in the field of view of the cameras. For 3D tracking applications a stereo camera rig has to be calibrated. The Zhang [1] method of the Octave/Matlab toolbox uses planar homography between a known calibration pattern (checkerboard) and semi-automatically digitalized images of the grid. To provide enough data to the algorithm we have to present to the cameras a large number of calibration squares of four infrared markers moving around in the three dimensions of the space (Fig. 1).

Fig. 1 Stereo camera rig and calibration square as seen by both cameras with the 4 infrared markers identified With the precisely reordered sets of the 2D coordinates collected in real time from each camera, we generate a calib_data.mat file for

123

each camera without manual intervention. A pre stereo-calibration is automatically triggered after acquisition and computes the intrinsic and extrinsic parameters of the sensors (Fig. 2). By mean of triangulations, we approximate unequivocally the 3D coordinates of each marker seen by both cameras. In a second step, a batched real time pattern acquisition of around five hundred images filtered by 3D distances measurements helps to refine the calibration. To address eventual noise issues, an implementation of Kalman and particule filters has been considered.

Fig. 2 Illustration of the extrinsic parameters after refined stereo calibration Results Different tests with several geometrical and environmental constraints, like the use of sterile bags, were performed after the calibration. It showed astonishing accuracy in 3D localisation. Infrared markers arranged in different 3D patterns gave a trivial mean to differentiate objects or devices and trigger specific actions. Conclusion This method helped to perform a fast-refined 3D calibration of a stereo Wiimote camera rig while taking advantages of the pros and getting around the limitations of the hardware. It opened many exploration paths in the world of augmented realities and 3D tracking with the help of affordable mainstream products and open source applications. References [1] Zhang Z, ‘‘Flexible Camera Calibration By Viewing a Plane From Unknown Orientations’’, Computer Vision The Proceedings of the Seventh IEEE International Conference on, 666–673 vol. 1, 1999. [2] Bouguet J-Y, ‘‘Camera calibration toolbox for matlab,’’ 2004–2015.

Development of a visualization and quantitative assessment system of laparoscopic surgery skill based on trajectory analysis from USB camera image T. Yamaguchi1, K. Suzuki1, T. Sugino1, R. Nakamura2

Int J CARS 1

Chiba University, Graduate School of Engineering, Chiba, Japan Chiba University, Center for Frontier Medical Engineering, Chiba, Japan

2

Keywords Instrument detection  Motion analysis parameters  Quantitative assessment  Visualization Purpose Minimally invasive surgery, such as laparoscopic surgery, has become popular, but also sophisticated. Hence, surgeons require regular training to improve their surgical techniques. Quantitative assessment and visualization of a surgeon’s skills are important to improve training outcomes. To assess a surgeons’ performance, motion analysis parameters of a surgical instrument must be obtained automatically. In our previous study, we developed a quantitative assessment system of operation performance using motion analysis parameters (MAPs) of the instrument using a surgical navigation system [1]. However, this system is expensive and complex, thus rendering it unsuitable for daily use. In order to solve this problem, an inexpensive and simple USB camera can be used. In this study, we present a method for automatically detecting the instrument position from the USB camera images through image processing and calculation of the instrument trajectory to determine the MAPs. These parameters can then be used for quantitative and multilateral assessment and visualization of surgical performance. Methods For automatic detection of the instrument by image processing, we use background subtraction and Hough transformation because the instrument is a moving, linear object. Figure 1 shows our proposed method. First, we obtain an edge image before and during a task, and then create a subtraction edge image using these two images. Next, we perform a Hough transformation on the subtraction edge image. This image is referred to as the straight-line detection image. By performing these two steps, we prevent the detection of the straight line coming from other background objects. At the same time, the moving objects are detected by a background subtraction process. This image is referred to as the background subtraction image. Using the above process, we can detect the instrument and it alone by the combined straight-line detection image and background subtraction image. Now, we calculate a trajectory from the position of the instrument tip detected in the previous method and then calculate the MAPs. We use task time, instrument cross time, work area size (rectangle and ellipse), work density, and average velocity/acceleration/relative velocity as MAPs. Instrument cross time is the time during which the left-hand instrument tip coordinate xl is larger than the right-hand instrument tip coordinate xr. Our strategy is to define the rectangular area in which the instrument is performing by following three steps. First, we calculate multiple rectangles that contain over 70 % of all the instrument tip points. Next, we determine the smallest rectangular areas of those from the first step. Finally, we calculate a rectangle that contains all of the rectangles defined in the second step. This final rectangle is defined as the rectangular work area. An elliptical work area is defined as the furthest exterior approximate oval of contour lines calculated from the density function of a two-dimensional Gaussian distribution. The work density is the number of points within 50 pixels from the center point of the work area in which surgeons should operate. For performance assessment and visualization, we extract the MAPs in which a significant difference is confirmed between the group having long and short laparoscopic histories, and we look for the MAP combination for which there is no multicollinearity. In this step, we can obtain various MAP combinations that do not correlate with each other. Next, we perform a principal component analysis (PCA) of these MAPs and extract the primary and secondary principal components.

By performing a clustering analysis with these principal components as each surgeon’s score, we achieve a visualization and quantitative assessment of the surgeon’s overall performance. Moreover, we define a performance factor and target value to evaluate each individual MAP. The mean of each MAP for all subjects is A, the standard deviation of each MAP for all subject is S, and the mean of each MAP of surgeons who have long laparoscopic surgery history is AE. The target value G is calculated as: G ¼ ð10  ðAE  AÞ=SÞ þ 50

Fig. 1 Instrument detection method We calculate G for all MAPs. When one surgeon’s MAP score is v, the surgeon’s performance value V is calculated as: V ¼ ð10  ðv  AÞ=SÞ þ 50 We calculated V for all MAPs. By displaying both the target value and the surgeon’s performance value in the same radar chart, users can quantitatively and intuitively grasp when an MAP is different from the target value. Therefore, in this study, we selected the suturing task for performance assessment because it is one of the basic laparoscopic surgery skills. We performed an MAP selection experiment for the overall assessment and individual assessment of surgeon performance. Additionally, we conducted a visualization and quantitative assessment of surgeons’ overall skill using the MAPs from 42 laparoscopic surgeons. Results We found a significant difference in task time, instrument cross time, rectangular work area of both hands, average velocity of both hands, average acceleration of the right-hand, and average relevant velocity. As there is no multicollinearity in each parameter, we select these eight parameters as MAPs. Figure 2 shows the clustering results of the primary and secondary principal components extracted by the PCA. We defined the expert as one who has a laparoscopic surgery history of more than 10 years, the intermediate as one who has a laparoscopic surgery history of 5 to 10 years, and the novice as one who has a laparoscopic surgery history of less than 5 years. As Fig. 2 shows, our method can classify estimated experts (outlined by dashed lines) and estimated intermediates (outlined by dotted lines). Some novices were classified as intermediate, which suggests that our proposed method could visualize skill features without including the years of experience. Hence, the score extracted by the PCA may have reflected the surgeons’ skill. In addition, we launched the radar chart of individual MAPs. The radar chart is considerably different between each group, allowing a surgeon to grasp the difference between the target values quantitatively and intuitively.

123

Int J CARS

Fig. 2 Clustering result Conclusion We developed an instrument detection method and performed a visualization and quantitative assessment of surgical performance using MAPs. A trajectory calculated from the instrument tip position using our proposed instrument detection method could have sufficient accuracy to calculate MAPs. Our proposed clustering method could classify estimated experts, intermediates, and novice surgeons. Furthermore, the analysis indicated that some surgeons cluster into a group that is different from their classification based on years of experience. Additionally, our proposed visualization method of surgeons’ MAPs obviously indicated a difference between each group. These results suggest that our scoring and visualization method could assess skill levels inconsistent with years of experience and is a considerably effective tool for use in understanding surgical skills. References [1] Sugino T et al. (2014) Surgical task analysis of simulated laparoscopic cholecystectomy with a navigation system. Int J of CARS, 9(5):825–836.

Intra-operative three dimensional ultrasound reconstruction and visualization for endoscopic liver surgery S. Onogi1, T. Ikeda2, J. Arata3, R. Nakadate1, S. Oguri4, T. Akahoshi5, K. Harada6, M. Mitsuishi6, M. Hashizume1,5 1 Kyushu University, Center for Advanced Medical Innovation, Fukuoka, Japan 2 Kyushu University Hospital, Department of Advanced Medicine and Innovative Technology, Fukuoka, Japan 3 Kyushu University, Faculty of Engineering, Fukuoka, Japan 4 Kyushu University, Innovation Center for Medical Redox Navigation, Fukuoka, Japan 5 Kyushu University, Faculty of Medical Sciences, Fukuoka, Japan 6 The University of Tokyo, School of Engineering, Tokyo, Japan

echogram. To address the issue, we have developed both a robotic hand for LUS [1] and a navigation system. In this study, we describe the navigation system which provides intraoperative 3D ultrasound by scanning LUS based on a freehand 3D ultrasound technique [2]. Methods The system consists of an ultrasound imaging system (Arietta 60, Aloka) with a LUS transducer (L43 K, Aloka), an electro-magnetic (EM) position sensor (Aurora, NDI), and a Windows workstation with in-house software developed by using C ++ language. A transducer position measured by the EM sensor and the US image are transferred to the workstation, then a 3D ultrasound volume is reconstructed and visualized instantly. As for the software implementation, computational costs are quite important for intra-operative use. Moreover, 3D volume reconstruction should be performed as well as its rendering. GPGPU is a possible solution to achieve the fast computation. As preparation of the volume reconstruction, volume size and position are determined, and output volume and temporal buffer (2-channel volume for voxel intensity and weight) are allocated in GPU memory. Reconstruction is achieved by the following steps: (1) a captured US image and positional information (transformed matrix) are transferred to a GPU memory, (2) each voxel position is transformed to the image space, (3) the pixel value is picked up by bi-linear interpolation, which is generally implemented in texture memory of GPU. And, Gaussian weight is computed from the distance among the voxel and the image plane. (4) The pixel value and weight are added to each channel of the temporal buffer. (5) the voxel value is obtained as the quotient of the buffer channels. The step (2)-(5) are performed in parallel for each voxel by GPU. The output volume data can be directly used as 3D texture for volume rendering without redundant memory copy among CPU-GPU. To confirm the feasibility of the developed software system, an experiment using a polyurethane liver phantom made from a patient CT data was conducted. The phantom contains both the artery and vein, and has well acoustic properties for experiments using an echography. Results Figure 1(a) shows a mimic endoscopic view using a video camera. An EM sensor was attached on the transducer. Figure 1(b) is a raw echogram of the phantom. In Fig. 1(c), a 3D model of the transducer and reconstructed volume are visualized in 3D space. The volume reconstruction and rendering are processed in each frame during scan. The computational time of both the reconstruction and visualization was approximately 50 ms. The time is less than typical imaging period of ultrasound; therefore, the system can reconstruct and visualize 3D ultrasound without drop frames when the framerate of ultrasound is less than 20 frames per second. Figure 2 shows the reconstructed volume, and vessel structure was successfully reconstructed intraoperatively.

Keywords Laparoscopic ultrasound  Volume reconstruction  Endoscopic liver surgery  Surgical navigation Purpose In laparoscopic liver surgery, surgeons require high skills and experiences for accurate and safe procedures. They generally use laparoscopic ultrasound (LUS), which can be inserted in abdominal cavity via a 12 mm port. The LUS is helpful to understand inner structure of an organ such as vessels and tumors; however, conventional LUS is not convenient due to poor operability. Moreover, it is hard to understand the relationship between a surgical view and an

123

Fig. 1 The validation test setup and result. (a) mimic endoscopic view using a video camera, (b) echogram of the phantom, and (c) navigation view with the transducer model, the echogram plane, and reconstructed volume

Int J CARS

Fig. 2 Reconstructed 3D volume obtained by the system Conclusion In this study, we developed the ultrasound navigation system for endoscopic liver surgery. The system can provide 3D volume intraoperatively in real-time. The fast processing is very important for intraoperative use because the imaging is frequently needed during surgery. In future, accuracy validation and in vivo feasibility test are planned to provide better support of laparoscopic liver surgery. References [1] Oguri S, Arata J, Ikeda T, Nakadate R, Onogi S, Akahoshi T, Mitsuishi M, Hashizume M (2015) Hand held manipulator that can robustly perform ultrasound scan in laparoscopic surgery. The 11th Asian Conf on Comput Aided Surg (ACCAS 2015), Singapore. [2] Onogi S, Wu J, Yoshida T, Masuda K (2015) Patient-mounted Robot for 2D Ultrasound Probe Scanning using McKibben Artificial Muscles. Advanced Biomedical Engineering 3, pp. 130–138.

Methods The procedure splits up into three steps: Acquisition of depth information from multiple images, validation of obtained 3D structures by comparing several measurements and finally iterative improvement of the 3D model. Depth is acquired by computing disparity information between image pairs using H. Hirschmu¨ller´s semi-global block matching technique [2]. Other concepts for deriving depth information from images can be used alternatively. Validation of computed depths is performed by fusing disparity maps in a single reference view. The idea behind this fusion step is adapted from Merrell et al. [3] where the authors formulate visibility-relations between corresponding depth values to find for each pixel of the fused mapthe most stable disparity value. The availability of multiple corresponding values for each pixel allows estimation of how reliable the selected values are. Each fused disparity map along with its reliability map is then used in the final stage to update the 3D model. To ensure convergence of the model after a finite number of iterations, a confidence map is saved. If several successive fused disparity maps support the model, confidence values are converging and the model is complete. Finally, a scheme based on image pyramids is applied to enforce smoothness of the model. Each layer of the pyramid represents the model in a different level of detail. The confidence map is utilized to estimate disparity values for each pyramid layer: Values with a higher confidence are weighted higher than those with less confidence. Starting at the highest layer, each value in the subjacent layer is adapted to fit in the range spanned by values of neighbouring pixels of the corresponding pixel in the upper layer. Results The system precision was evaluated by comparison of the reconstructed model to a ground truth that was provided by a handheld structured light scanner with precision up to 40 nm. In the three experiments a standard deviation of the point clouds of 3 to 5 mm after 4 to 6 iterations was achieved. An iterative closest point algorithm was used to align the ground truth to the model at each iteration of the reconstruction procedure. The computed transformation and rotation of the ground truth point cloud converged to values of 1 to 5 mm and 0.001 to 0.02 (Figs. 1, 2).

3D human face reconstruction for surgical navigation M. Katanacho1, J. Hoffmann1, S. Engel1 1 Fraunhofer Institute for Production Systems and Design Technology IPK, Medical Systems Engineering, Berlin, Germany Keywords Dense surface reconstruction  Surgical navigation  Disparity images  Depth map fusion Purpose Tracking the position and orientation of a surgical instrument enables enhanced intra-operative visualization for the surgeon. This aids surgical planning and decision making and thus benefits minimally invasive surgery. Already existing approaches are based on optical or electromagnetic tracking systems which have the drawbacks of additional required space and line-of-sight issues. Novel methods aim at utilizing modern navigation algorithms to perform surgical navigation only with a standard video camera attached to an instrument [1]. These techniques require a precise 3D model of the observed scene in order to accurately determine the instrument´s position. In this paper a reconstruction scheme that takes advantage of a continuous input stream of images with known camera positions to form a surface model of a human face is presented.

Fig. 1 The three stages of the reconstruction procedure. Top: Two disparity images computed by semi-global block matching. Middle: Fused disparity image (left) and reliability map (right). Bottom: Final model (left) and confidence map (right)

123

Int J CARS Image guidance for improving electrode placement precision in EEG study S. Jeon1, J. Chien1, J. Song1, J. Hong1 1 DGIST, Robotics Engineering, Daegu, South Korea Keywords EEG electrode placement  International 10–20 system  Anatomical landmark  Surgical navigation

Fig. 2 Reconstruction results. Top: Ground truth (left) and model (right). Bottom: Difference in depth [mm] The main source of error in the reconstruction procedure arises from false measurements of the camera´s position together with the factor of high measurement sensitivity since the camera is very close to the reconstructed scene. Conclusion The presented 3D reconstruction system proves to yield good results even under unfavourable conditions (misleading positional measurements). The developed algorithm is able to shape a model of a scene over several iterations and thus compensates sporadic errors. The applied smoothing scheme preserves local details while enforcing global consistency of the model. Acknowledgement This work is funded by the German Federal Ministry of Education and Research (BMBF), research grant 13GW0039B. References ¨ zbek C., Kosmecki B., [1] Katanacho M., Engel S., Winne C., O Keeve E., 2015. Visuelle Navigation—Systemkonzept fu¨r die navigierte Chirurgie. 14. Jahrestagung der Deutschen Gesellschaft fu¨r Computer- und Roboterassistierte Chirurgie (Mu¨nchen). [2] Hirschmu¨ller, H., 2008. Stereo Processing by Semiglobal Matching and Mutual Information, in Pattern Analysis and Machine Intelligence. IEEE Transactions on, vol. 30, no. 2, pp. 328–341. [3] Merrell, P., Akbarzadeh, A., Liang Wang, Mordohai, P., Frahm, J.-M., Ruigang Yang, Nister, D., Pollefeys, M., 2007. Real-Time Visibility-Based Fusion of Depth Maps. ICCV 2007. IEEE 11th International Conference on, vol., no., pp. 1–8, 14–21.

123

Purpose In neurorehabilitation of patients with disorders, such as stroke and cerebral palsy, patient assessment is crucial not only to evaluate the degree of motor function impairment, but to perform effective intervention as a part of the process of recovery. Advancement of functional neuroimaging technologies, including functional near infrared spectroscopy (fNIRS) and electroencephalography (EEG) has enabled such evaluations in order to observe changes in cortical activity in the motor area of interest during the rehabilitation. Of them, EEG is particularly proper due to its high temporal resolution and relatively dense scalp coverage. Since, in general, rehabilitation is conducted repeatedly over a long period of time, repeatability of EEG electrode placement is one of the most important prerequisites to obtaining reliable EEG signals. Conventional methods for EEG electrode placement have relied on the international 10–20 system or its expanded systems like the 10–10 and 10-5 system [1]. Those methods first partition the head surface by using several anatomical landmarks as fiducial points, including nasion, inion, left and right preauricular points and then place the electrodes on lattice points generated by the partition. However, manual identification of the landmarks via visual perception and palpation induces huge variations in their determined locations for every trial, thus highly degrading the repeatability of the electrode placement [2]. To address the issue, we propose a method to reliably place EEG electrodes using real-time image guidance, which is cost-effective and does not require repetitive landmark identifications for every electrode placement. Methods The proposed method uses a vision tracking system using a monocular webcam (Logitech HD Pro Webcam C920, Logitech, Lausanne, Switzerland) for tracking locations of the patient and EEG cap, and a three-dimensional (3D) laser scanner (Artec Sider, Artec Group, California, USA) for generating 3D surface models of them. The tracking system was developed to localize the patient and EEG cap, or the affixed markers by identifying unique patterns printed on them and calculating their pose using the identified information with the perspective n-point algorithm implemented in an open-source computer vision library (OpenCV, Intel, California, USA). First the patient wears the EEG cap based on the 10–20 system and scanning is performed. Several predefined electrode locations were then measured both in the model coordinate frame of the scanner and in the real-world coordinate frame of the tracking system. Following the measurement, two sets of point correspondences are established: One for model-to-patient and the other for model-to-cap. With pairedpoint registration method [3], the former is used for mapping between the real world and the virtual model space and the latter for matching orientation of the electrodes on the virtual model space and those tracked in the real world as shown in Fig. 1. Following the setup process, the location of the EEG cap relative to the head is displayed in the 3D virtual space that is provided by an open-source visualization platform, 3D Slicer (Brigham & Women’s Hospital, Boston, USA) as shown in Fig. 2.

Int J CARS

[2]

[3]

[4]

positioning systems.’’ Neuroimage, Vol. 34, No. 4, pp. 1600–1611, 2007. Atcherson SR, Gould HJ, Pousson MA, Prout TM ‘‘Variability of electrode positions using electrode caps.’’ Brain topography, Vol. 20, No. 2, pp. 105–111, 2007. Arun, Somani K, Huang TS, Blostein SD ‘‘Least-squares fitting of two 3-D point sets.’’ Pattern Analysis and Machine Intelligence, IEEE Transactions on 5, pp. 698–700. 1987. Zhang Z ‘‘Iterative point matching for registration of free-form curves and surfaces.’’ International journal of computer vision, Vol. 13, No. 2, pp. 119–152, 1994.

Development of an ultra-high-definition camera for real-time multi-view tracking of a handheld surgical robot

Fig. 1 Related transformations for electrode navigation

I. Verdu1, S. Hohnstein1, M. Vetter1 1 Hochschule Mannheim University of Applied Sciences, EMB-Lab, Mannheim, Germany Keywords Multiview  Tracking  Handheld-robot  FPGA

Fig. 2 Concept of electrode navigation. (left) during the navigation. (right) after navigation From the next trials, electrode placement can be easily and precisely accomplished without relying on the 10–20 system, once simple laser scanning and fiducial localization steps are conducted to compensate for the disparity in poses of the head marker between the first reference trial and the current trial via iterative closest point surface matching [4]. Results To evaluate the accuracy and precision of the proposed electrode placement method, electrodes were placed on the patient head using the proposed method and the conventional method. The placement errors for each method were measured on ten predefined electrode locations over 20 repeated trials. The placement error was mm for the proposed method and mm for the conventional 10–20 system, respectively, which shows the significantly reduced mean placement error and standard deviation of the proposed method. Conclusion This study proposed a method to reliably place EEG electrodes using real-time image guidance. Using the proposed method, one who does not have a priori knowledge on the 10–20 system is expected to precisely place EEG electrodes, without manually identifying ambiguous anatomical landmarks. Acknowledgments This work was supported by the DGIST R&D Program of the Ministry of Science, ICT and Future Planning (16-BD-0401). References [1] Jurcak V, Tsuzuki D, Dan I ‘‘10/20, 10/10, and 10/5 systems revisited: their validity as relative head-surface-based

Purpose The Intelligent Tool Drive (ITD) [1] is a handheld robot developed to provide assistance in surgical procedures. This device is meant to be held by a surgeon during bone machining interventions, and its position relative to the bone is to be automatically calculated using a combination of optical tracking and inertial sensors. The performance of the ITD is heavily influenced by the tracking system’s precision and update speeds, which demands that the camera modules be able to record high-resolution video at high frame rates [2]. Moreover, these modules must act as autonomous video processing components that can be connected to form a scalable multi-camera network, which will ultimately be able to record an arbitrary volume [3] (i.e. the tracking volume) and provide this information in real time during the robotsupported surgery. Each camera is responsible for acquiring the video data, performing the low-to-mid level image processing algorithms and transmitting the results to a ‘Concentrator’ unit, which consolidates the data from all the cameras and, together with the inertial sensor information, estimates the position of the robot. Figure 1 shows the overall architecture of the tracking system.

Fig. 1 ITD’s Tracking System Architecture In this work, a functioning prototype of the required camera modules is developed. More specifically, the hardware and software components needed for the real-time acquisition and processing of a 25 Megapixels (5120 9 5120p) video stream at 50 frames per second (FPS) are designed, verified and implemented on an FPGA. The programmable device is then connected to an existing platform which holds the image sensor, power management circuitry, communication interfaces, etc., to form a complete working system.

123

Int J CARS Methods The main physical components of the system are the image sensor and the FPGA. For the former, a monochrome VITA 25 K CMOS image sensor from ON Semiconductor was used. For the latter, a Xilinx Zynq-7000 All-Programmable SoC (XC7Z020) was selected, which in addition to programmable logic includes a dual-core Cortex A9 processing system implemented as hard IP. Figure 2 shows the architecture of the image processing pipeline within the camera.

Fig. 2 Image Processing Pipeline Architecture The hardware modules developed in the FPGA are divided in four subsystems [4]. First, the Sensor Interface subsystem is responsible for a) configuring the image sensor and controlling its operation via an SPI and b) acquiring the image and synchronization information transmitted by the sensor over the LVDS channels and generating parallel pixel streams to be processed by the downstream logic. Second, the Region Extraction subsystem detects active regions within the frame (i.e. objects in the foreground) and stores their runlength encoding (RLE) representation using on-chip memory resources. Third, the DMA Bridge subsystem reads the extracted RLE data and transmits it to the system’s main memory over DMA, from which it can be accessed by the processor for further analysis. Finally, the Video Display subsystem subsamples the raw video data and generates a video stream that is suited to be displayed in a standard monitor using an HDMI interface. In addition to the custom logic modules, a fitting clock structure was implemented using the on-chip clocking resources. Several software functions were developed in order to obtain a fully-functional system. These include low-level drivers for controlling the hardware modules, including the SPI and the synchronization/ deserialization of the incoming LVDS data, as well as for configuring several operational parameters at run time. Likewise, the functional verification of all hardware components was carried out through RTL simulations before being implemented on the FPGA. Results After completing the design and verification phase, the system was implemented on the actual hardware, and the image acquisition was carried out. As test setup, an active marker was placed in front of the camera, and the recorded images were fed to the acquisition and processing pipeline. After the information was extracted and stored to the main memory, it was read back by the processing system, which proceeded to calculate the center of gravity of a subset of the acquired data. Finally, the results were transmitted to the Concentrator unit, which used this data to calculate the position of the marker. This allowed the system-level functionality to be verified, confirming that the entire data acquisition and processing pipeline was functioning properly. In addition to this system-level test, the behavior of individual modules was checked using on-chip debugging cores, which provided visibility into the implemented design for optimization and bug-fixing purposes. The video processing pipeline within the camera was implemented as eight parallel video streams, each one

123

corresponding to a vertical slice of the image (640 9 5120p) clocked at 248 MHz. Conclusion The developed system met the real-time video acquisition and processing specifications of the target application. The vertical slicing of the image allowed the video processing pipeline to be clocked at a manageable frequency, while still being able to carry out all the processing within the active line time. Because the feature extraction is performed in a per line basis, the row overhead time is used to transmit the extracted information to main memory, minimizing the system latency. From a system’s perspective, the image processing pipeline implemented in hardware enables a significant reduction of the amount of data to be analyzed by the software-based components, thus helping the Concentrator unit meet its real-time execution requirements. Further work should focus on implementing more sophisticated tracking algorithms in the programmable logic, in order to maintain a low system latency between the data acquisition and the delivery of the tracking information. This work was supported in part by the BMBF under the grant 13EZ1205D, Intelligent Tool Drive. References [1] Intelligent Tool Drive Project. Bundesministerium fu¨r Bildung und Forschung (BMBF) Fo¨rder-kennzeichen: 13EZ1205D. [2] Hohnstein S, Stereo optical tracking for medical applications: Novel tracking system to track needle shaped navigation aids. Master thesis, Hochschule Mannheim University of Applied Sciences. Mannheim, Germany. November 2010. [3] Schu¨lein P, Multi-view Kamera Tracking fu¨r medizinische Anwendungen. Analyse von Tracking- und Kalibrierungsalgorithmen hinsichtlich Genauigkeit und Kameraplatzierung. Master Thesis, Hochschule Mannheim University of Applied Sciences. Mannheim, Germany. November 2015. [4] Verdu I, Development of an FPGA-based Platform for RealTime Acquisition and Processing of Ultra High Definition Video Tracking Data. Master Thesis, Hochschule Mannheim University of Applied Sciences. Mannheim, Germany. November 2015.

Towards a hybrid tracking and navigation for the integrated intervention environment

system

B. Glass1, B. Kraus1, M. Vetter1 1 Mannheim University of Applied Sciences, EMB-Lab, Mannheim, Germany Keywords Navigation  Data fusion  Optical tracking  Inertial tracking Purpose The efficient execution of assisted interventions depends to a great extent on the integration of different medical, diagnostic and therapeutic processes and methods. The necessary degree of integration has not yet been reached as of today and this commonly inhibits the use as well as conceptual research and development of molecular intervention procedures. In the scope of the research project M2OLIE (Mannheim Molecular Intervention Environment) the EMB-Lab is working on a hybrid tracking system and navigation concept to address common problems in integrated intervention environments regarding working volume, sample rate, latency and obstruction of sensors. Different object groups such as surgical tools, assisting robots, and imaging devices as well as the patient and medical professionals in the operating room have different requirements regarding the performance of the tracking and navigation solution. We propose to

Int J CARS

partition the intervention room according to these requirements into groups which combine different tracking modalities and apply data fusion methods to provide an improved navigation solution. Tracking modalities will comprise marker-based and marker-less methods such as high-precision optical systems with multiple cameras, inertial tracking devices, Time-Of-Flight cameras, volume scanners and active wireless tags. The objective here is to combine the individual tracking modalities for the following data fusion step according to the requirements of an individual object group. In this paper we want to showcase our work in progress for the tracking concept using hybrid data fusion methods [1]. As an example we chose the case of tracking the environment directly surrounding the patient which has the highest requirements regarding the performance of the tracking system. Methods According to the requirements analysis, we define three principal tracking groups arranged concentric around the patient. The innermost group (A) contains the patient, tools, robots and the medical professionals directly involved in the intervention. The next group (B) contains the extended volume of the interventional setting such as imaging devices and displays/interfaces, inactive robots and auxiliary medical equipment. The outermost group (C) extends to the physical limits of the intervention room and encompasses the full working volumes of robots, all present personnel and specially defined zones like keep-out areas. The groups as well as the working volume of the different tracking modalities are partially overlapping. Group C has the lowest requirements regarding latency and precision but the largest working volume, therefore we propose the use of volume scanners to monitor the zones of the operating room in combination with active wireless tags that do not require a constant line-of-sight and provide status information. Group B contains objects that are auxiliary to the intervention with varying requirements towards latency and precision. A major aspect here is the realization of user interfaces and identification as well as tracking of personnel position. Here we propose the use of Time-OfFlight sensors and marker-based optical tracking. The actual intervention process takes place in group A with the patient at the center. The use of assisting robot systems demands a very low latency and high precision as well as reliability from the tracking solution. Therefore we propose the use of a hybrid tracking system. We use an optical tracking system which provides high accuracy and precision, but has low sample rate and high latency as well as reliability issues due to obstruction of marker structures. To compensate these drawbacks we propose the addition of inertial sensors to the optical markers which provide low latency and high sample rates without line-of-sight requirements, but lack the absolute accuracy of the optical system [2]. Research on the fusion of the optical and inertial sensors is carried out using an Extended Kalman Filter approach (Fig. 1) [3], where the high-rate inertial sensor output is preprocessed in a strap-down process model. The output of this model is updated in intervals with the lower sample rate of the highly accurate optical tracking system. In the time between updates, the EKF acts as a predictor for the position and orientation of the tracked object (Fig. 2).

Fig. 1 Schematic of the hybrid sensor fusion Extended Kalman Filter

Fig. 2 Example of the fusion filter position output Results To verify the tracking concept and hybrid data fusion methods proposed in this paper the model for the fusion of optical and inertial sensor data has been implemented in Matlab. Simulations were carried out on artificial data. Further experiments were conducted with sensor data obtained from using a commercial optical tracking system and a hybrid marker developed at the EMB-Lab. The results show that the fusion EKF is able to successfully compensate for the optical system’s low sampling rate and latency. Conclusion The proposed concept and methods were shown to have achieved reasonable performance. The implemented system provides a basis for further research for the fusion of different tracking modalities for objects with different requirements. Further work is also necessary on the topics of handling the hand-over between object groups and tracking modalities as well as incorporating reliability and quality information provided by either the sensors directly or a model of the process.

123

Int J CARS This work was supported in part by the BMBF under grant 13GW0090A, Forschungscampus M2OLIE, project M2INT. References [1] Oh HM, Kim MY ‘‘Attitude tracking using an integrated inertial and optical navigation system for hand-held surgical instruments,’’ in Control, Automation and Systems (ICCAS), 2014 14th International Conference on, vol., no., pp. 290–293, 22–25 Oct. 2014. [2] Tobergte A, Pomarlan M, Passig G, Hirzinger G ‘‘An approach to ulta-tightly coupled data fusion for handheld input devices in robotic surgery,’’ in Robotics and Automation (ICRA), 2011 IEEE International Conference on, vol., no., pp. 2424–2430, 9–13 May 2011. [3] Tobergte A, Pomarlan M, Hirzinger G ‘‘Robust multi sensor pose estimation for medical applications,’’ in Intelligent Robots and Systems, 2009. IROS 2009. IEEE/RSJ International Conference on, vol., no., pp. 492–497, 10–15 Oct. 2009.

Desktop 3D printing in medicine to improve surgical navigation in acral tumors V. Garcı´a-Va´zquez1, G. Rodrı´guez-Lozano1,2, R. Pe´rez-Man˜anes1,2,3, J. A. Calvo1,2,3, D. Garcı´a-Mato4,5, M. Cuervo-Dehesa2,3, M. Desco4,5,6, J. Pascau4,5,6, J. Vaquero1,2,3 1 Instituto de Investigacio´n Sanitaria Gregorio Maran˜o´n, Madrid, Spain 2 Hospital General Universitario Gregorio Maran˜o´n, Servicio de Cirugı´a Ortope´dica y Traumatologı´a, Madrid, Spain 3 Facultad de Medicina. Universidad Complutense de Madrid, Departamento de Cirugı´a, Madrid, Spain 4 Universidad Carlos III de Madrid, Departamento de Bioingenierı´a e Ingenierı´a Aeroespacial, Madrid, Spain 5 Instituto de Investigacio´n Sanitaria Gregorio Maran˜o´n, Madrid, Spain 6 Centro de Investigacio´n Biome´dica en Red de Salud Mental (CIBERSAM), Madrid, Spain Keywords Acral tumors  Navigation  3D printing  Mold Purpose Image guidance may improve limb-sparing surgery results. For this purpose, preoperative computed tomography (CT) or magnetic resonance (MR) images can be registered to patient’s anatomy by means of a tracking system. In this approach, a rigid transformation is commonly applied for the registration step. This assumption may not be correct for acral tumors because distal extremities have many joints with complex movements (e.g. hand has 27 degrees of freedom). For reducing target registration error (TRE), a similar limb position must be ensured in the preoperative images and during image guided surgery (IGS). Additive manufacturing and rapid prototyping are easily available for clinical use thanks to three-dimensional (3D) printing. In orthopedic surgery, patient-specific anatomical models have been created for several purposes such as preoperative planning, pre-bending plates design, training and patient-physician communication. Patient-specific 3D surgical aids are used to improve the surgical planning and assist the surgeon during the procedure [1]. Regarding desktop 3D printing applied to orthopedic surgery, good results have recently been documented in reconstructive pelvic surgery [2] and in

123

techniques of lower-limb realignment [3]. In this context, we propose to take one step further using a desktop 3D printer to design and create patient-specific distal extremity molds that ensure a similar position during imaging and IGS in the operating room (OR). The aim of this abstract is to describe a new workflow that merges IGS and desktop 3D printing in limb sparing surgery of distal extremities. Our study also evaluates the reproducibility of distal extremity position during navigation using a patient-specific 3D-printed hand phantom. Methods The first step in the suggested workflow for IGS in acral tumors starts by printing a 3D mold of the distal extremity. This holder will lay on the surgical bed allowing the limb to remain in a fixed and known position. The mold cannot be too tight since pressure may cause edema to the patient. The holder is modeled by extruding a patient’s hand surface created from the segmentation of a previous CT image. Alternatively, structured-light 3D scanners could be used in this step, thus avoiding ionization radiation. Several conical holes (Ø 4 mm x 3 mm depth) are modeled on the mold surface to be used as landmarks for the pointer in the registration step. Moreover, three screws (designed to attach optical passive markers) are included in the layout to define a reference frame that would allow accounting for mold movements during navigation. The modeling step is done with freely available software (MeshMixer and 123D Design, Autodesk, Inc., USA). The desktop 3D printer used is Witbox-2 (BQ, Spain), a lowcost fused deposition modeling (FDM) hardware managed with opensource software. The thermoplastic material chosen is polylactic acid (PLA) because of its extrudability and nontoxic properties. Furthermore, the resulting holder can be sterilized by ethylene oxide [4]. The next step involves the surgical planning (segmentation of tumor and definition of surgical margins) using the radiological postprocessing software Horos (GNU open-source OsiriX,www.horosproject.org) on previous CT or any CT/MR study acquired with the limb placed on the printed mold. Conical holes can be used to facilitate the registration between CT and MR images. Finally, during IGS, the distal extremity is placed on the sterilized mold. The navigation is performed with an optical tracking system (OTS) after registering the conical holes of the holder in the image space with those corresponding ones obtained with a tracked pointer in the OR (physical space). A patient with a soft-tissue sarcoma in the palm of his right hand was selected for evaluating the reproducibility of the limb placement on its mold during navigation. The holder (Fig. 1) was created from the CT used to plan the neoadjuvant external radiotherapy. A 3D model of the hand was also printed separately from the mold. This mold (Fig. 1) included nine conical holes (Ø 4 mm x 3 mm depth) on its surface for placing the pointer tip in the validation step. To resemble the patient’s CT image, a CT scan was acquired with the rigid hand on its mold (voxel size 0.5 9 0.5 9 0.5 mm). Navigation was performed with a multi-camera OTS (OptiTrack, NaturalPoint Inc., USA) with 7 cameras, which reduces occlusion problems caused by OR personnel and surgical support devices and has previously been evaluated [5]. The tracking system was connected by means of Plus Toolkit (www.assembla.com/spaces/plus/wiki) to a 3D Slicer platform (www.slicer.org) with a SlicerIGT extension. The evaluation consisted in placing the 3D printed hand on its mold, registering the conical holes (mold) of the CT (image space) with those corresponding ones obtained with a tracked pointer (physical space) and, finally, estimating the TRE between the conical holes (hand) in the image space and in the physical space. These steps were repeated four times.

Int J CARS

Fig. 1 Hand mold with three optical markers (left) and printed hand with the tumor (right) Results Table 1 shows the fiducial registration error (FRE) obtained from the conical holes in the mold and TRE from the conical holes in the printed hand. Figure 2 displays the physical space and the image space after registration.

Table 1 FRE and TRE for each repetition of the navigation process Repetition

FRE (mm)

TRE (mm)

Mean

Mean

RMSE

SD

Max

1

1.37

1.23

1.42

0.77

2.46

2

1.39

1.76

1.84

0.57

2.90

3

0.98

1.27

1.33

0.42

1.98

4

0.58

1.01

1.11

0.47

2.07

Total (mean)

1.08

1.32

1.43

0.56

2.35

Fig. 2 Navigation example (physical space [left] and image space [right]) Conclusion This study presents an IGS workflow for acral tumors that includes desktop 3D printing for reproducing distal extremity position. A multidisciplinary team of surgeons and engineers worked together in the process of modeling and printing the patient-specific mold with a low-cost FDM printer at the hospital. TRE was in accordance with a previous study using the same tracking system [5], demonstrating the reproducibility of hand position during navigation. These results allow us to follow this procedure during the final intervention that is scheduled in one month’s time. Results with real patient data will be presented during the conference.

Acknowledgments This work was supported by projects IPT-2012-0401-300000, TEC2013-48251-C2-1-R, DTS14/00192, PI15/02121, EU FP7 IRSES TAHITI (#269300) and FEDER funds. We would like to thank the Spanish company BQ for the donation of the 3D printing hardware for clinical use. References [1] Malik HH, Darwood ARJ, Shaunak S et al., J Surg Res, 199(2), 512–22, 2015. [2] Perez-Man˜anes R, Arnal-Burro´ J, Rojo-Manaute JM et al. 3D Surgical printing in preoperative planning and pre contoured plates for acetabular fractures. Do It Yourself. J Knee Surg [article in press]. [3] Perez-Man˜anes R, Rojo-Manaute JM, Vaquero Martin J et al. 3D surgical printing in open-wedge high tibial osteotomy. Do It Yourself. Injury [article in press]. [4] Rankin TM, Giovinco NA, Cucher DJ et al. 3D printing surgical instruments: Are we there yet? J Surg Res, 189(2), 193–7, 2014. [5] Garcı´a-Va´zquez V, Marinetto E, Santos-Miranda JA et al., Phys Med Biol, 58, 8769–82, 2013.

An experimental set-up for Navigated-Contrast-Agent and Radiation Sparing Endovascular Aortic Repair (Nav-CARS EVAR) M. Horn1, J.-P. Goltz2, J. Modersitzki3, N. Papenberg3, M. Schenk3, W. Schade4, M. Kleemann1 1 University Hospital Lu¨beck, Departement of Surgery, Lu¨beck, Germany 2 University Hospital Lu¨beck, Department of Radiology and Nuclear Medicine, Lu¨beck, Germany 3 Institute of Mathematics and Image Computing, Fraunhofer MEVIS Lu¨beck, Lu¨beck, Germany 4 Technical University Clausthal, Fraunhofer Heinrich-Hertz-Institute, Goslar, Germany Keywords Endovascular aortic repair  EVAR  Endovascular navigation  Fibre bragg navigation  Virtual angioscope Purpose Endovascular treatment of aortic aneurysm by stenting procedures (Endovascular aortic repair—EVAR) has medical benefits compared to the open surgery by a lower 30-day mortality, a faster recovery of the patients and shorter hospital stay. Over the last decade EVAR has been developed from single centre experiences to a standard procedure. With increasing clinical expertise and medical technology advances treatment of even complex aneurysms are feasible by endovascular methods. One integral part for the success of this minimally invasive procedure is innovative and improved vascular imaging to generate exact measurements and correct placement of stent prosthesis. One of the most difficulties in learning and performing this interventional therapy is the fact, that the threedimensional vascular tree has to be overlain with the two-dimensional angiographic scene by the endovascular specialist. Disadvantages of the EVAR-procedure are the use of nephrotoxic contrast agents and the exposure to radiation to the patient and the physician. Methods We report the development of real-time navigation software, which allows a three-dimensional endoluminal view of the vascular system during an EVAR-procedure in patients with infrarenal aortic aneurysm. Even in patients with complex anatomy, the surgeon or interventionalist achieves an accurate option of spatial perception concerning the current vascular anatomy, e.g. the position of the guide-wire. We used the preoperative planning CT angiography for

123

Int J CARS three-dimensional reconstruction of aortic anatomy by volume-rendered segmentation. At the beginning of the intervention the relevant landmarks are matched in real-time with the two-dimensional angiographic scene. Geometric differences between the two image data due to respiratory of the patient as well as the deformability of the vessels must be considered. During the intervention the software continuously registers the position of the guide-wire or stent-graft. An additional 3D-screen shows the generated endoluminal view during the whole intervention in real-time, including visualization of plaque anatomy and outgoing vessels. Results Our preliminary results of navigated endoluminal virtual angioscopy are promising. We examined the combination of hardware and software components including complex image registration and fiber optic sensor technology (fibre-Bragg navigation). The presented experimental navigation system for EVAR consists of 4 steps:

A microscope image overlay system with intra-operative automatically camera calibration for brain tumor resection

•

Purpose In an intelligent operating room with an open magnetic resonance imaging (MRI) system, surgeons resect tumors while checking the navigation system with intra-operative MR images. Brain tumors, blood vessels and surgical instrument on intra-operative MR images and the tip positional information of surgical instrument from threedimensional position measurement system are displayed on the navigation system in real time. Using this, surgeons can operate while checking the position of the tumors and blood vessels on the navigation system. However, it is difficult for surgeons to use accurate positional information because they compare the navigation monitor and the operative field. For solving this problem, image overlay system using augmented reality technology is effective. However, this system requires an exclusive microscope that can detect magnification and focus. In addition, this system requires additional working, including preoperative and intra-operative calibration [1]. It is difficult to use an exclusive microscope because the high costs and the additional working requirements impose a burden on the surgeons. Therefore, we propose a microscope image overlay system that can be applied using the existing microscope and does not require any additional working [2]. The proposed system calibrates the camera using two kinds of surgical instrument positional information automatically. Thereafter, the system overlays intra-operative MR images on a microscope-based display. After we developed the system, we evaluated the accuracy of image overlay using clinical data. Methods The proposed system is composed of a Brainlab navigation system (Curve, Brainlab AG), a computer (CPU: Intel(R) Core(TM) i7-4790, RAM: 16.0 GB OS: Windows 8.1 Pro) for image overlay, and an existing microscope (Fig. 1).

• • •

Fibre bragg tracking technology is used to continuously localization of position and curvature of the guide-wire or stent graft sheath. Integration of fibre optic sensors in stent graft introducer sheaths. Real time registration of fibre optic information into preoperative planning CT. Placement of stent graft sheath with included fibre optic sensor into patient-specific vascular phantoms in an experimental setting and visualization as virtual 3-dimensional angioscopy.

The patient-specific aortic models are produced by rapid prototyping using the ‘‘poly-jet technology’’. From a technical point of view the feasibility of fibre-Bragg navigation has been proven in our experimental setting with patient-based vascular models. Three-dimensional preoperative planning including registration and simulation of virtual angioscopy in real time are realised (Fig. 1).

S. Yamamoto1, I. Sato2, T. Noguchi1, Y. Fujino2, H. Yamada3,4, T. Suzuki3,5, Y. Muragaki3, K. Masamune3 1 Future University Hakodate, Graduate School of Systems Information Science Engineering, Hakodate, Hokkaido, Japan 2 Future University Hakodate, System Information Science Research, Hakodate, Japan 3 Institute of Advanced Biomedical Engineering and Science, Tokyo Women’s Medical University, Tokyo, Japan 4 Murakumo Corporation, Tokyo, Japan 5 Medical Device Strategy Institute, Japan Association for the Advancement of Medical Equipment, Tokyo, Japan Keywords Image overlay  Calibration  Navigation system  Microscope image

Fig. 1 Prototypic visualization of the endovascular navigated stentgraft implantation during EVAR procedure including virtual endoluminal view (‘‘Virtual Angioscope’’) Conclusion The aim of the Nav-CARS-EVAR concept is reduction of contrast medium and radiation dose by a three-dimensional navigation during the EVAR procedure. The ‘‘Virtual Angioscope’’ should improve intraoperative visualization, placement of guide-wires and stentgrafts. The prototype also offers the possibility of intervention planning and simulation which may lead to a reduced learning curve and therefore patient safety. To implement fibre-Bragg navigation further experimental studies are necessary to verify accuracy before clinical application.

123

Fig. 1 Configuration of the proposed system This system acquires surgical instrument positional information using the navigation and microscope images from the microscope. In addition, the system performs image processing and acquires surgical instrument positional information. Thereafter, the system calculates

Int J CARS

the camera calibration parameter automatically using two kinds of surgical instrument positional information from the navigation system and the microscope. Finally, the system overlays intra-operative MR images on a microscope-based display. This system overlays intraoperative MR images on a microscope image by the following four steps: (1) The system acquires surgical instrument positional information by Vector Vision Link (V.V. Link) from the navigation system. At the same time, the system acquires microscope images using a video capture unit from the microscope. (2) The system performs image processing in the acquired image (i.e., hue, saturation, and value [HSV], region of interest [ROI], Hough transform) and acquires tip positional information of surgical instrument. HSV: In the proposed algorithm, the system acquires microscope images of red, green, and blue (RGB) color space from the microscope. Then, the algorithm changes the images from RGB color space to HSV color space, and extracts surgical instrument’s handle. ROI: After extracting the surgical instrument’s handle, the algorithm designates ROI at the top of the handle. Hough transform: The algorithm detects straight lines in the ROI by hough transform, and we define intersection of the lines as a tip of surgical instrument. After this, the algorithm calculates the intersection surgical instrument positional information from the microscope images. (3) The system calculates the camera calibration parameter automatically using two kinds of surgical instrument positional information and calculates camera parameters (intrinsic and extrinsic camera parameters) for image overlay by the least-squares method. (4) The system overlays intra-operative MR images on a microscope-based display using calculated camera parameters. We evaluated the accuracy of automatically calibrated image overlay, changing the amount of acquired surgical instrument positional information. We acquired clinical data of brain tumor surgery using surgical instrument positional information and microscope images. At this time, we set amount of that information thirty points and eighty points. After this, the system overlays intra-operative brain tumors and blood vessels images on a microscope-based display using calculated camera parameters at each point. Then, we evaluated image overlay accuracy of the tumors and blood vessels on microscope images. Results The proposed system acquired the surgical instrument positional information from the navigation system and microscope images from microscope during brain tumor resection. Thereafter, the system overlaid intra-operative MR images on a microscope-based display (Fig. 2). As a result, the system was able to calibrate the camera automatically and overlay intra-operative MR images on a microscope-based display. We evaluated the accuracy of the image overlay error using thirty points and eighty points of surgical instrument positional information. The errors were 194 pixels at thirty points (Fig. 2A) and 126 pixels at eighty points (Fig. 2C). In addition, the system acquired thirty and eighty points from microscope images (Fig. 2B, D) and the amount of surgical instrument positional information that the system acquired was increased and the accuracy of image overlay improved as the surgery progressed.

Fig. 2 Results of image overlay at thirty and eighty points (A, C) and plots of thirty and eighty points at microscope image (B, D) Conclusion We developed a brain tumor resection microscope image overlay system with intra-operative automation camera calibration. This system can overlay intra-operative MR images on a microscope-based display without an exclusive microscope and additional working such as preoperative and intra-operative calibration. In addition, the accuracy of image overlay improved by increasing the amount of acquired surgical instrument positional information as the surgery progressed. The accuracy of image overlay when the system acquired eighty points was the best, such that the error was 126 pixels. Using this, surgeons can recognize the position of the tumors, blood vessels, and surgical instrument intuitively. Therefore, we believe the operations will be performed with higher accuracy in neurosurgery. References [1] Giraldez TJG, et al. (2007) Design and Clinical Evaluation of an Image-Guided Surgical Microscope with an Integrated Tracking System.’’ International Journal of Computer Assisted Radiology and Surgery, 1:253–264. [2] Takahashi H, et al. (2015) A Neurosurgical Microscope ImageOverlay System with Intra-operative Real-Time Camera Calibration. International Journal of Computer Assisted Radiology and Surgery 1:S50–51.

Port placement planning assistance for laparoscopic gastrectomy based on anatomical structure analysis Y. Hayashi1, K. Misawa2, K. Mori1 1 Nagoya University, Information & Communications, Nagoya, Japan 2 Aichi Cancer Center Hospital, Department of Gastroenterological Surgery, Nagoya, Japan Keywords Laparoscopic surgery  Port placement  Surgical planning system  Stomach Purpose This paper presents a port placement planning assistance method for laparoscopic gastrectomy. Laparoscopic gastrectomy is widely

123

Int J CARS performed as minimally invasive procedure for early gastric cancer in Japan [1]. In laparoscopic surgery, surgeon operates a target organ using a forceps while watching a laparoscope monitor. These instruments are inserted into the abdominal cavity through the ports placed on the abdominal wall. Relationships between the ports’ locations and the target anatomical structures’ locations lead to ease surgical operation. Therefore it is important to decide the optimal port placement in laparoscopic surgery. Several research groups have reported on an optimal port placement method for robotic surgery [2, 3]. This paper proposes a method for optimal port placement planning assistance for laparoscopic gastrectomy. Methods In laparoscopic gastrectomy, a surgeon usually places five ports. A port placed umbilicus is usually used for inserting a laparoscope. Two ports placed on the right side of the patient are used for a main operator, and two ports placed on left side of the patient are used for an assistant surgeon. In this study, we focus on the method for determining the port location for the main operator. Since the anatomical orientation of the blood vessels is very important for laparoscopic gastrectomy, we identified the relationship between the target blood vessels’ location and ports’ location as the condition for determining the optimal port placement. The right gastric artery (RGA) and the left gastric artery (LGA) are selected as target anatomical structures. According to the surgeon’s point of view and their practices, we introduce two angle conditions for determining the optimal port placement: (C1) angle between forceps and laparoscope on each blood vessel and (C2) angle between forceps and horizontal plan on each blood vessel. These two angles are defined using the midpoint of the two ports for the main operator. The C1 is defined as arccosðððp12  vi Þ  ðp3  vi ÞÞ=ðjp12  vi jjðp3  vi ÞjÞÞ, and the C2 is defined as arccosðððp12  vi Þ  nÞ=ðjp12  vi jjnjÞÞ, where p12, p3, vi, and n are the midpoint of the ports, the position of the laparoscope port, the position of the vessel i, the normal of the horizontal plane. The horizontal plane is determined as the plane defined by the patients’ the right-to-left and the head-to-foot directions. We consider that the placement of the ports within a certain definite range of these angles makes surgical operations to the blood vessels easier. We utilize four angle conditions, the C1 on RGA, the C2 on RGA, the C1 on LGA, and the C2 on LGA, to determine the optimal midpoint locations. The ranges for angle conditions are determined based on the port locations decided by the experienced surgeons during the laparoscopic gastrectomy. These port locations are measured in the operating room using Polaris Spectra (NDI, Waterloo, Ontario, Canada). We also measure the blood vessels’ positions on CT images. Also the horizontal plane is defined based on CT images. The positional information of the ports and the positional information of the blood vessels have different coordinate systems. To calculate the ranges using these data, we perform the patient-toimage registration to align between the coordinate systems [4]. Results We obtained the positional information during laparoscopic gastrectomy in 22 cases. We calculated the averages and the standard deviations of the angle conditions using the obtained positional information in 18 cases. The averages and the standard deviations of the C1 on RGA, the C2 on RGA, the C1 on LGA, and the C2 on LGA are 37.0 ± 4.7, 33.2 ± 3.5, 50.8 ± 7.3, 46.3 ± 5.6 degrees, respectively. Remaining four cases are used for the evaluation. We calculated the average difference between the average angles computed in 18 cases and the angles computed in four cases. The average differences in the C1 on RGA, the C2 on RGA, the C1 on LGA, and the C2 on LGA are 3.9, 3.4, 5.6, and 6.4 degrees, respectively. The standard deviations of angle conditions are small. Furthermore, the average differences between the average angles in 18 cases and the angles in four cases, which do not include 18 cases, are also small. These results show these angles are stable for determining the port placements by the experienced surgeons. We consider that these

123

angles are suitable criterion for determining the optimal port placement. We visualized the areas of the midpoint location of the optimal port using the angle conditions. The ranges for each angle condition are determined using the positional information in 18 cases. Figure 1 shows the visualization results in volume rendering images of CT images. In this figures, the regions surrounded by the cones indicate the midpoint location of the optimal port. Surgeons can determine the port locations using these assistance images. Especially, this information helps the untrained surgeons to decide the port placement. We believe that the proposed method is very useful for performing laparoscopic gastrectomy.

Fig. 1 An example of areas of optimal port locations. Areas enclosed by cones shows optimal port areas determined by angle conditions Conclusion This paper presented a port placement planning assistance method for laparoscopic gastrectomy based on anatomical structure analysis. The proposed method utilized the angle conditions to determine the optimal port placement. We also generated the port placement determination assistance images using the angle information. These images help surgeons to decide the port placement in laparoscopic gastrectomy. References [1] Etoh T, Inomata M, Shiraishi N, Kitano S (2013) Minimally invasive approaches for gastric cancer -Japanese experiences, J Surg Oncol 107: 282–288. [2] Adhami L, Coste-Maniere E (2003) Optimal planning for minimally invasive surgical robots, IEEE Trans Rob Autom 19: 854–863. [3] Cannon JW, Stoll JA, Selha SD, Dupont PE, Howe RD, Torchiana DF (2003) Port placement planning in robot-assisted coronary artery bypass, IEEE Trans Rob Autom 19: 912–917. [4] Hayashi Y, Misawa K, Oda M, Hawkes DJ, Mori K (2015) Clinical application of a surgical navigation system based on virtual laparoscopy in laparoscopic gastrectomy for gastric cancer, Int J CARS, Online First Articles.

Cochlear implant dissection simulator for training pediatric otolaryngology surgeons B. Reilly1, D. Preciado1, H. Sang2, A. Jain1, A. Enquobahrie3, S. Arikatla3, K. Cleary1

Int J CARS 1

Children’s National Health System, Washington, United States Tianjin Polytechnic University, Tianjin, China 3 Kitware Incorporated, Carrboro, United States 2

Keywords Simulation  Cochlear implant  CT  Haptic rendering Purpose Cochlear implantation (CI) is the standard of care for infants born with severe hearing loss. Cochlear implants provide otherwise deaf children the ability to integrate into the auditory world and ultimately achieve mainstream speech and language function. Current FDA guidelines approve the surgical placement of implants as early as 12 months of age. Implantation at a younger age poses a greater surgical challenge, yet is associated with improved language outcomes. The underdeveloped mastoid tip, along with thin calvarial bone, creates less room for surgical navigation and can result in increased surgical risk. We aim to develop a CI dissection simulator for training surgeons implanting young infants. Methods The simulator system will be developed based on pre-procedure CT images from pediatric infant cases (\12 months old) at our hospital. The simulator will include segmentation of the key structures, an algorithm for converting the current position of the surgical drill to a simulated force, a haptic interface for providing force feedback, and a 3D display that will simulate the microscope views from the procedure. A system block diagram is shown in Fig. 1.

effector are computed in the simulation engine and are communicated to the haptic device. This is accomplished by using position control algorithms that are part of the device API. The haptic forces are updated at a higher rate of around 1000 Hz to achieve high fidelity rendering. The position and orientation of the HMD are also communicated to the simulation software in order to update the visual frames. This is performed at 60 Hz in order to maintain the 3D perception. The cochlear surgical anatomy is shown in Fig. 2, representing a 3D rendering (right) from the pre-operative temporal bone CT (left) scan of an 8 month old patient who underwent CI at our center. Key structures, such as the round window position and bony overhang are depicted in the figure (red arrow). The simulator will particularly focus on this as well as the location of the facial nerve relative to the angle of the round window membrane, the aeration of the mastoid, and the vector of the cochlear basal turn (not shown).

Fig. 2 Cochlear surgery anatomy Conclusion In this abstract we presented the design and our initial work on a simulator for pediatric cochlear implant dissection. The next steps are to build the system and evaluate it using input from our attending surgeons in otolaryngology.

An initial approach for learning surgical declarative knowledge G. Dardenne1, B. Labbe´1, C. Chesneau1, J.M. Diverrez1, P. Jannin1, 3, 4, L. Riffaud2, 3, 4 1 Institute of Research and Technology, b\[com, Rennes, France 2 Department of Neurosurgery, Pontchaillou University Hospital, Rennes, France 3 INSERM, U1099, Rennes, France 4 LTSI, Universite´ Rennes 1, Rennes, France Fig. 1 Block diagram of simulator

Keywords Surgical process models  Surgical training  Declarative knowledge  Knowledge modeling

Results Figure 1 represents the overview of the simulator whose main components are the simulation engine, a head mounted device (HMD such as the Oculus Rift) and the custom designed haptics device. A human user interacts with the simulator interfacing with the HMD and the haptic device thus forming a closed loop system. The simulation engine is the software that is composed of physics based simulation and rendering algorithms in order to furnish visual and haptic feedback to the human user. The inputs to the simulation engine are the position and orientations of the HMD and the end effector of the haptic device which are in turn controlled by the user. A custom haptic device will be designed to hold a surgical drill similar to the one used in actual implant surgery as the end effector. The haptic device shown in the figure has 6 degrees of freedom in motion and 3/6 degree of freedom in haptic rendering. The forces resulting from the interactions of the virtual representation of the end

Purpose Surgical training has always been a topic of interest mainly based on two knowledge categories: declarative knowledge, which can be characterized as ‘‘knowing what’’, and procedural knowledge, which can be characterized as ‘‘knowing how’’ [1, 2]. For a novice, it’s however essential to acquire both before being able to practice. Today, there is a lack of formal approaches and tools allowing, first, an expert surgeon to record and formalize surgical declarative knowledge, and second, a novice surgeon to have an easy access to this knowledge. Methods We propose an approach and a software application based on Surgical Process Models (SPM) [3] allowing an expert to structure the surgical declarative knowledge and a novice surgeon to make easier the training phase. The main view of the software application is shown in Fig. 1. The surgical procedure is presented as a sequence of phases

123

Int J CARS and steps located along a vertical line. Information and data can be added by the expert and visualized by the learner following the different steps of the surgical workflow.

Fig. 1 Software interface with knowledge associated to the steps of the SPM We evaluated the concept with six neurosurgeons for a dedicated neurosurgical procedure: the endoscopic third ventriculostomy (ETV). Specific surveys have been filled before and after interaction with the software. These surveys were mainly based on two different approaches: the UTAUT method for the assessment of the usage intention [4] and the ATTRAKDIFF method for the evaluation of hedonic and pragmatic qualities of a new product [5]. Both approaches allowed us to evaluate the software using specific ergonomics indicators: utility, acceptability and usability. Results According to the UTAUT results, the software application is perceived as easy to use. The users think that it will allow them to be more efficient in their work and intend to use this tool in the future. The Fig. 2 shows the ATTRAKDIFF score. According to the answers, this tool is perceived as simple, clear, manageable, original, innovative, stimulating, new, professional, presentable, pleasant, nice, and attractive, but also as not captivating and boring.

Participants had a positive perception concerning the utility of such software. Today, neurosurgeons have access to several resources (such as clinical and scientific papers) in order to learn declarative knowledge. However, this task, without a dedicated tool, can be tedious and long since the amount of data can be very huge. This tool is therefore perceived as an innovative tool allowing them to centralize information, about a surgical procedure, which have been defined and validated by experts. Concerning the acceptability, the answers observed from the ATTRAKDIFF survey indicated that the participants had a positive experience about this tool, but they also perceived it as not captivating and boring. This was mainly due to the fact that the learner was maybe relatively passive during the training. Concerning the usability, users think that the interface was well organized and structured. These feedbacks highlighted the fact that surgical process models could be well adapted for the structuration and the visualization of the data. The number of participants in this study was however limited. More participants need to be included in the future in order to get general conclusions about the real interest of using such tool in a daily practice. However, the advantages have been clearly highlighted, and especially concerning the use intention in the future. In conclusion, these preliminary results have shown that the concept and the software application could be an essential tool to: •

Structure and maintain information associated to a specific surgical procedure; • Make easier the access, for non-expert surgeons, to information relative to a specific surgery; • Optimize the declarative knowledge training phase associated to a given surgical procedure. References [1] Witte TEF (2015) Requirement for efficient robotic surgery training. Master thesis. University of Twente. [2] Van Merrie¨nboer JJG (1997) Training Complex Cognitive Skills: A Four-Component Instructional Design Model for Technical Training. Educational Technology Pubns. [3] Lalys F, Jannin P (2014) Surgical process modelling: a review. International Journal of Computer Assisted Radiology and Surgery. doi:10.1007/s11548-013-0940-5. [4] Venkatesh V, Morris M, Davis F, Davis G (2003) User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly. 27:425–478. [5] Hassenzahl M, Burmester M, Koller F (2003) AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualita¨t. Mensch & Computer. 57:187–196.

Deformable resection process map for estimating appearance of vascular structures in cutting procedures

local

M. Nakao1, K. Taura2, T. Matsuda1 1 Kyoto University, Graduate School of Informatics, Kyoto, Japan 2 Kyoto University Hospital, Department of Hepato-Biliary-Pancreatic and Transplant Surgery, Kyoto, Japan Keywords Geometry estimation  Deformation  Virtual resection  Intraoperative navigation Fig. 2 Attrakdiff score for the developed software Conclusion We proposed an approach for learning declarative knowledge. This approach was evaluated in the context of a neurosurgical procedure with six neurosurgeons. Initial results showed the feasibility and the interest of such approach and the corresponding software.

123

Purpose Virtual planning using preoperative CT/MR images allows quantitative, strategic planning of patient-specific cutting procedures for tumor resection. The planned cutting path and virtual organ images are used as intraoperative cutting guides [1]. During surgery, however, the limited visible parts of the vascular structures (e.g. optically visible or measured from imaging devices such as ultrasound), are

Int J CARS commonly used for intraoperative decision making rather than the preoperative plans. One reason to abandon the virtual plan is that the shape of the virtual organs often differs from the deformed states of real organs. A deformed shape may be due to altered physical condition (e.g. air/blood pressure), push/pull manipulation or cuts made during a surgical procedure. The clinical requirements for estimating the visual appearance of such local features are increasing in order to perform evidence-based cutting and reduce surgical risks. Although efforts have been made to provide multilateral anatomical information for navigating cutting procedures [1, 2], the local appearance of vascular structures in the intraoperative deformed state has been omitted in planning/navigation software due to the difficulty of modeling the effects of soft tissue cuts. In this presentation, we introduce the deformable Resection Process Map (RPM) for estimating local appearance of vascular structures after cuts as a novel guide for soft tissue tumor resection procedures. The deformable RPM is a geometrical estimator that provides a time-varying local map based on the deformed geometry of the organs (see Fig. 1). Unlike static virtual-reality-based training simulators for cutting/ablation procedures, we designed a set of algorithms to provide a semi-automatic software framework tuned for planning/navigation. The RPM can be directly generated from patient-specific medical images using volumetric resampling techniques.

Fig. 1 The deformable resection process map as a guide for soft tissue tumor resection procedures Methods To achieve semi-automatic generation of the RPM from medical images, we have newly designed an objective function f(I, pk) that computes the smooth cutting path S from the segmented organ image I and a set of cutting points pk. First, the three-dimensional organ region is sparsely sampled and a proxy geometry bounded by the reference cutting path S0 and enclosing the sampled points is generated. The proxy geometry can be described using a tetrahedral mesh. When some cutting points pk are given on the vascular structures or the organ surface, the vertices of the proxy geometry are relocated using a quadratic minimization function, which is designed to preserve the local shape consistency of the given points pk and the reference cutting path S0. The cutting points can be manually supplied by the user or extracted from the boundary of segmented blood vessel regions. The estimated cutting path S locally fit to the cutting points in least squares manner. The curved shape of S can be more complex as the indicated cutting points increase. Next, the discontinuous deformation around the cutting path is computed for the proxy mesh and rendered using the tetrahedral volume rendering algorithm [3]. The vascular structures in the deformed body are linearly interpolated and visualized volumetrically in the proxy geometry. This approach estimates the cutting path using the given cutting points as geometrical constraints and produces the time-varying local map of vascular structures by progressive deformable representation. Results We have applied the software framework to 21 CT datasets for hepatectomy and generated the RPMs in a variety of the deformed

states. When some cutting points are indicated on the organ surface or vessels, the RPM is updated and the virtual organ is deformed by applying external force to the cut surfaces. Figure 1 demonstrates a time-series representation of the resection process map generated from a single CT image set. As the procedure progresses, an updated local appearance is rendered of anatomical structures such as a tumor, hepatic vein, and portal vein in the deformed organ, and a cross sectional image of vascular structures already split by cutting. We have also confirmed that real-time updating of the cutting path and rendering of the deformed organ at greater than 20 frames/s was possible on general-purpose computers with graphic processing units (CPU: 3.5 GHz, Memory: 8 GB, GPU: NVIDIA GeForce 780). We note that the geometry update is performed by the objective function f without relying on vertex addition or mesh subdivision. This scheme enables fast and real-time computation while handling the time-varying geometry of the cutting path. This concept addresses technical issues discussed in [2] and formulates the RPM as a generalized computation framework that can be applied to nonanatomical cutting paths by improving volumetric resampling techniques [4]. Conclusion We have introduced the deformable resection process mapping as a time-varying geometric guide for cutting procedures. The algorithmic design for semi-automatic generation from medical images was described. Because user input of some cutting points is the only requirement for generating the RPM, the developed software will be directly available for clinical use in previewing surgical procedures and intraoperative workflow management without time-consuming setup or additional work loads. Quantitative evaluation of the generated cut surface geometry and clinical validation are our future work. References [1] Takamoto T, Hashimoto T, Ogata S, Inoue K, Maruyama Y, Miyazaki A, Makuuchi M (2013) Planning of anatomical liver segmentectomy and subsegmentectomy with 3-dimensional simulation software. American Journal of Surgery 206 (4):530–538. [2] Lamata P, Lamata F, Sojar V, Makowski P, Massoptier L, Casciaro S, Ali W, Studeli T, Declerck J, Elle OJ, Edwin B (2010) Use of the resection map system as guidance during hepatectomy. Surgical Endoscopy 24 (9):2327–2337. [3] Hung KWC, Nakao M, Yoshimura K, Minato K (2011) Background-incorporated volumetric model for patient-specific surgical simulation: A segmentation-free, modeling-free framework. International Journal of Computer Assisted Radiology and Surgery 6 (1):35–45. [4] Nakao M, Oda Y, Taura K, Minato K (2014) Direct volume manipulation for visualizing intraoperative liver resection process. Computer Methods and Programs in Biomedicine 113 (3):725–735.

A wearable intraoperative stereoscopic fluorescence imaging system with hand-held microscopy and ultrasound C.A. Mela1, F. Papay2, W.K. Thompson3, Y. Liu1 1 University of Akron, Biomedical Engineering, Akron, OH, USA 2 Cleveland Clinic Foundation, Dermatology & Plastic Surgery Institute, Cleveland, OH, USA 3 John H. Glenn Research Center, Cleveland, OH, USA Keywords Fluorescence  Imaging  Intraoperative  Ultrasound Purpose Intraoperative imaging modalities can be utilized to great effect during cancer surgeries. Methods such as MRI, CT, ultrasound and

123

Int J CARS fluorescence imaging have been implemented intraoperatively. Previously, we have reported on the development and use of a Stereoscopic Intraoperative Imaging Goggle [1, 2]. The goggle system was developed to provide the surgeon with real-time, wide-field stereoscopic fluorescence imaging in a wearable display unit. Alternative to conventional 2D monitor based display systems, our goggles incorporation of hands-free line-of-sight imaging with the depth perception provided by stereoscopic vision better simulates natural perception. This makes the system highly intuitive and reduces the need for the surgeon to correlate the fluorescence information with the surgical landscape. We have since improved upon our original designs by incorporating multimodal capabilities to the goggle system, integrating both a hand-held fluorescence microscope as well as ultrasound imaging. We have also further developed a new multi-sensor model which registers full color images with the fluorescent data in real time. These advances will provide doctors with more tools to aid in a variety of diagnostic and surgical procedures assisted by the Integrated Imaging Goggles. Methods Fluorescence imaging has been conducted using medically relevant FDA-approved dyes (Indocyanine Green (ICG) and Protoporphyrin IX (PPIX)) (Sigma Aldrich, USA). The system is comprised of two CCD imaging sensors with adjustable lenses and appropriate optical bandpass filters (Edmunds Optics, USA). Each imaging sensor is placed in-line with the line of perception of the user, providing the user with a natural, line-of-site stereoscopic imaging effect, Fig. 1A. The interpupillary distance can be adjusted to better fit the user. Fluorescence excitation was provided by a filtered variable intensity illumination module. Both the imaging sensors and the display modules were connected to a laptop via USB, for power and processing.

Fig. 1 A. 2-sensor version goggle for wide-field stereoscopic fluorescence imaging. B. Wide-field fluorescence imaging of fluorescent tissue (blue arrow) with picture-in-picture displaying microscopic (purple arrow) information. The microscope image is utilizing a colormap to better visualize the fluorescent data. The orange arrows indicate simulated satellite tumors. C. Wide-field fluorescence imaging of ICG injected into a chicken breast (blue arrow) with picture-in-picture displaying ultrasound (purple arrow) information. The orange arrow indicates a foreign body detected within the tissue. D. Fluorescence imaging (blue arrow) co-registered with color reflectance for improved fluorescent target localization and anatomical data visualization The custom hand-held fluorescence microscope was connected to the computational module via USB for image integration. Using this

123

device, the surgeon can get a real-time, magnified view of a small fluorescent target which can aid in the identification of residual or satellite tumors. The microscopic imagery has been integrated directly into the display seen by the surgeon, Fig. 1B, allowing for the simultaneous viewing of both the wide-field stereoscopic images as well as the microscopic image. Additionally, a portable USB connectable ultrasound machine (Ultrasonix, China) has been incorporated into the imaging system, Fig. 1C. This affords the surgeon the ability to gain useful depth information while still conducting fluorescence imaging. To provide the surgeon with more anatomical data, we have designed another imaging goggle which combines color reflectance imaging with fluorescence, through the addition of two color-imaging CMOS sensors placed directly above the fluorescence imaging sensors. Results Using the goggle system, we have achieved a minimum fluorescent detection limit (SBR [ 2) at a 40 nM concentration of either dye while operating at a frame rate of 30 fps. We have found the microscope to achieve a fluorescent detection limit of 60 nM (SBR [ 2). Future upgrades to the system will improve upon system sensitivity and resolution. We have been able to resolve objects, as seen through the goggle display, under 0.5 mm in any dimension for working distances of up to 50 cm. The system has been tested in simulated surgical operations, during which a fluorescent tissue was identified and excised from chicken tissues. Conclusion We have incorporated fluorescence microscopy and ultrasound with wide-field, line-of-sight stereoscopic fluorescence imagery in an easy to use wearable system. Stereoscopic imaging systems have been demonstrated to be of significant benefit in guiding tumor resections. Binocular vision allows for significantly improved depth perception, helping a surgeon distinguish the 3 dimensional shape or the depth of a tumor while also improving hand-eye coordination. Additionally, faster surgical times and reduced number of false positives have been reported when using 3D medical imaging systems [3]. We have leveraged upon these advantages in creating the Integrated Imaging Goggle, whose compact size, light weight and low cost will make it an appealing option in a variety of clinical settings. Acknowledgements This study was supported in part by grants from the Ohio Department of Development (TVSF 15-0123), the Leading Entrepreneurial Academics into Practice Award (F14-10-LIU-WEB), the National Aeronautics and Space Administration (NNX14AL37H), and the University of Akron Startup Funds. References [1] Mela CA, Patterson C, Thompson WK, Papay F, Liu Y (2015) Stereoscopic Integrated Imaging Goggles for Multimodal Intraoperative Image Guidance. PLoS ONE 10(11): e0141956. [2] Mela CA, Patterson C, Papay F, Liu Y (2015) Stereoscopic optical imaging goggle for guiding surgeries. Int J CARS 10(Suppl 1):S240–1. [3] Getty DJ, Green PJ (2007) Clinical applications for stereoscopic 3-D displays. SID 15(6):377–84.

Int J CARS

Poster Session 18th International Workshop on Computer-Aided Diagnosis

123

Int J CARS Pulmonary nodule classification with 3D features of texture and margin sharpness

Table 1 continued Features

All

Margin

Texture

Selected

Accuracy

0.763

0.76

0.764

0.785

AUC

0.809

0.785

0.803

0.801

Keywords Pulmonary nodule  Image classification  Image feature extraction  Pattern recognition

k=7

Accuracy

0.766

0.777

0.762

0.751

AUC

0.815

0.797

0.814

0.789

Purpose The classification of pulmonary nodules may be a complex task to radiologists due to temporal, subjective and qualitative aspects. Therefore, it is important to integrate computational tools to the early nodule classification process, since they have the potential to characterize objectively and quantitatively the lesions and to classify them without the necessity to wait for days. The goal of this work is to evaluate the malignant-benign classification of pulmonary nodules using a machine learning classifier and selected image descriptors of texture and margin sharpness. The proposed evaluation relies on five steps: the development of a nodule database; the extraction of 3D image features; the selection of the most relevant attributes from the feature vector; the classification of the pulmonary nodules in malignant or benign; and its performance assessment. Methods The pulmonary nodule image database used in this work has computed tomography (CT) scans provided by the Lung Image Database Consortium (LIDC) [1]. Each LIDC’s specialist was asked to assess the nodule’s malignancy likelihood. For the purposes of this work, nodules with high or moderate probability for malignancy were considered malignant (total of 426), and nodules with high or moderate probability for benignity were considered benign (total of 745). The feature vector has 48 3D attributes, which were normalized in a range 0–1. The following 36 texture attributes were extracted from a 3D version of the gray level co-occurrence matrix in orientations 08, 458, 908 and 1358: energy, entropy, inverse difference moment (IDM), inertia, variance, shade, promenance, correlation and homogeneity [2]. The following 12 margin sharpness attributes were extracted from perpendicular lines drawn over the borders on all nodule slices: difference of two ends, sum of values, sum of squares, sum of logs, arithmetic mean, geometric mean, population variance, sample variance, standard deviation, kurtosis measure, skewness measure and second central moment [3]. The feature selection method employed was a Wrapper. The search method was the Best First. We used the k-nearest neighbors (knn) to classify the cases and k values of 1, 3, 5, 7 and 9. Classification and feature selection were performed by a 10-fold cross-validation. Each nodule was characterized by four feature vectors: (1) all 48 attributes combined; (2) 12 margin sharpness attributes; (3) 36 texture attributes; and (4) selected attributes using a threshold value greater than 4 occurrences. The parameters of accuracy and area under the ROC curve (AUC) assessed the classification efficiency. Results Table 1 presents the results of the pulmonary nodule classification with different k values and subsets of features. No statistically significant difference was identified between results with 95 % confidence interval.

k=9

Accuracy

0.763

0.776

0.761

0.792

AUC

0.827

0.801

0.821

0.816

J. Ferreira Jr1, M. C. Oliveira2, P. M. Azevedo-Marques1 1 USP, FMRP, Ribeirao Preto, Brazil 2 UFAL, IC, Maceio, Brazil

k=5

Table 1 Pulmonary nodule classification efficiency

k=1 k=3

Features

All

Margin

Texture

Selected

Accuracy

0.723

0.709

0.722

0.751

AUC

0.698

0.687

0.699

0.645

Accuracy AUC

0.754 0.777

0.751 0.761

0.743 0.778

0.747 0.742

123

Feature selection improved classification accuracy when k was defined as 1, 5 and 9. Considering k as 1, selected attributes (promenance at orientations 08 and 908) obtained highest classification accuracy, with a mean increase of 3 percentage points in comparison to all features. However, highest AUC was obtained by texture attributes. Considering k as 5, selected attributes (shade and IDM at orientation 08 and correlation at 908) obtained highest classification accuracy, with a mean increase of 2 percentage points in comparison to all features. However, highest AUC was obtained by all attributes. Considering k as 9, selected attributes (IDM at 08 and 1358 and correlation at 908) obtained highest classification accuracy, with a mean increase of 3 percentage points in comparison to all features. Results also showed that they obtained highest classification accuracy (0.792) overall scenarios of the evaluation. However, highest AUC was obtained by all attributes, which also was highest AUC obtained overall scenarios (0.827). Further experiments were performed with higher values of k, but classification accuracy decreased as the k value increased and thereby we stopped performance evaluation at 9 nearest neighbors. Conclusion Reducing the dimensionality of the feature vector increased pulmonary nodule classification in the majority scenarios. For instance, using only 3 attributes instead of all 48 features (reduction of 94 % of the feature space) increased classification performance in 3 percentage points with k as 9. Our methodology presented worse results when comparing to the majority of the results found in literature using different image attributes and classifiers. However, in [4] the authors obtained 0.772 of AUC using support vector machines and 46 features, and this work obtained 0.816 of AUC with the aforementioned scenario. Our feature vector is in a developing stage, and we aim at improving its performance by extracting more relevant attributes and employing more robust feature selection algorithms and classifiers. References [1] Armato III S, McLennan G, Bidaut L, McNitt-Gray M, Meyer C, Reeves A et al. (2011) The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A Completed Reference Database of Lung Nodules on CT Scans. Medical Physics 38:915–931. [2] Haralick R, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Transactions on Systems, Man and Cybernetics 6:610–621. [3] Ferreira Jr J, Oliveira M (2015) Evaluating Margin Sharpness Analysis on Similar Pulmonary Nodule Retrieval. Proceedings of the 28th IEEE International Symposium on Computer-Based Medical Systems (IEEE CBMS) 60–65. [4] Reeves A, Xie Y, Jirapatnakul A (2015) Automated pulmonary nodule CT image characterization in lung cancer screening. International Journal of Computer Assisted Radiology and Surgery 1–16.

Int J CARS Proposal for a novel CAD development technique using artificially created case images K. Abe1, H. Takeo1, Y. Kuroki2, Y. Nagai3 1 Kanagawa Institute of Technology, Electrical and Electronic Engineering, Atsugi, Kanagawa, Japan 2 Kameda Kyobashi Clinic, Radiology, Chuo-ku, Tokyo, Japan 3 Higashisaitama National Hospital, Radiology, Hasuda, Saitama, Japan Keywords Liver tumors  CAD  Artificial case images  CT Purpose Research and development of computer aided diagnosis (CAD) for various areas of the body are currently underway. In the design of detection and classification processes, CAD development typically requires a considerable number of case images rich in variations for use as learning data for classifier devices. However, it is extremely difficult to procure the diverse case images required for learning. To deal with this, we are working on artificially creating case images by embedding case shadows (malignant tumors, etc.) into lesion-free images. The objective is to augment the sample data used in CAD system development by innumerably generating simulated case images that incorporate artificial tumors while varying their size, shape and contrast. In this study, we propose a novel CAD development technique that uses artificial case images. The use of artificial case images enables creation of diverse cases of any type, enabling the development of CAD capable of handling more complex cases. We verified the effectiveness of the new technique by comparing the performance of CAD systems developed using artificial case images with a system developed by conventional means of using only genuine images. Methods (1) Creating artificial case images Artificial case images are created using as materials CT images of cases with tumors and normal CT images. Artificial case images are created by embedding tumor images into normal CT images [1]. Figure 1 shows a general diagram of this process. A case image is artificially created by cutting tumor areas cut out of case images and embedding the tumor into a normal image using Poisson blending [2] for image composition. This method allows the tumor to be detected, its embedding position in the normal image, and its contrast, size, and other factors to be freely set.

Figure 2 shows the procedure for hepatic tumor CAD processing. A method of detection from CT values based on time phase differential images was used for the detection of candidate hepatic tumor areas. Using contrast medium to focus on time phase changes in a mass, differential images are created from plain CT and portal vein CT. Binarization is then performed from the density histograms of low and high density areas, and detection of candidate hepatic tumor areas carried out. Morphological opening is then used to remove micro areas. For classification processing, we used classifiers that employ SVMs. Multiple feature values of detected candidate areas were calculated, SVM machine learning executed, and classifiers designed. Adopting 11 feature values, including spheroidicity, major and minor axes and their length ratios, tumor volume to surface area ratio, and average and dispersion of pixel values of the tumor overall, classification is carried out in an 11 dimension vector space.

Fig. 2 Liver tumor CAD flow chart

Fig. 1 Diagram of hepatic tumor embedding Considering application to hepatic tumor CAD, this study we created artificial case images of hepatic tumors. (2) Hepatic tumor CAD

Results Based on a total of 100 cases, 20 TP (true positive) areas and 80 FP (false positive) areas, detected from hepatic tumor candidate areas, 5 types of classifiers were designed that mixed genuine and artificial cases at specified TP area case percentages and comparison carried out. Classifier A used genuine cases only, classifier B had 25 % of TP cases replaced by artificial cases, classifier C had 50 % of TP cases replaced by artificial cases, classifier D had 75 % of TP cases replaced by artificial cases, and classifier E used artificial cases for all TP cases. The classification performance of each classifier was evaluated using test data comprising genuine cases. The candidate hepatic tumor areas of the test data used comprised 10 TP areas and 40 FP areas. Table 1 shows the performance evaluation results for classifiers A to E.

123

Int J CARS Table 1 Performance of each classifier on unknown data TP [%]

FP [%]

Classifier A (Genuine 100 %, Artificial 0 %)

80

Classifier B (Genuine 75 %, Artificial 25 %)

70

7.5

Classifier C (Genuine 50 %, Artificial 50 %)

80

7.5

Classifier D (Genuine 25 %, Artificial 75 %)

60

5

Classifier E (Genuine 0 %, Artificial 100 %)

0

0

10

In the evaluation using test data shown in Table 1, classifiers designed with a certain amount of artificial case images mixed in showed no large differences in performance compared with the classifier designed only with genuine cases. To investigate the quality of artificial case image CAD learning, we also performed the following test. A total of 90 genuine cases, 10 TP cases and 80 FP cases, were used in the design of classifier X and test data evaluated. This was compared with test data performance for classifiers A and C. Table 2 shows a comparison of classifier X, designed with 90 cases, with classifiers A and C, each of which have 10 more learning cases (TP areas). Both exhibited about the same rate of performance improvement. Because performance rates match regardless of whether genuine cases are increased or artificial cases are increased, this suggests that the quality of artificial case images in CAD learning is equivalent to genuine case images. Table 2 Comparison with presence/absence of artificial case images FP areas are all kept constant at 80 cases

TP [%]

FP [%]

Classifier X (Designed with 10 genuine cases of TP 70 areas)

5

Classifier A (Classifier X + 10 genuine cases of TP 80 areas)

7.5

Classifier C (Classifier X + 10 artificial cases of TP 80 areas)

7.5

Conclusion In this study, we proposed a novel CAD development technique that uses artificial cases images. Classifiers with genuine cases images and artificial case images mixed in at specified ratios were created, and performance evaluations of the classifiers using test data were carried out. The results demonstrated that performance equivalent to CAD developed with genuine-case-only classifiers can be obtained with CAD developed with classifiers with a certain amount of artificial case images mixed in. Although the verification results of this study apply only to the single example of hepatic tumor CAD, this technique shows great promise in delivering the same result for CAD for other areas of the body. Going forward, it will be important to verify CAD development that uses artificial case images of CAD for other areas and evaluate its effectives. References [1] Porter T, Duff T (1984) Compositing digital images. ACM SIGGRAPH Computer Graphics, Vol. 18, pp. 253–259. [2] Perez P, Gangnet M, Blake A (2003) Poisson Image Editing. Proc. SIGGRAPH’03, pp. 313–318.

123

Usefulness of a z-score-based analysis of the temporal horn volume of the lateral ventricle for detection of early Alzheimer’s disease on CT images N. Takahashi1, T. Kinoshita1, T. Ohmura1, Y. Lee1, E. Matsuyama1, H. Toyoshima1 1 Research Institute for Brain and Blood Vessels-Akita, Radiology, Akita, Japan Keywords z-Score  Voxel-by-voxel analysis  Alzheimer’s disease  CT Purpose Rapid increase in Alzheimer’s disease (AD) patients has become a critical issue worldwide. Early diagnosis of AD is important and widely considered as one of the principal goals for AD care. Neuroimaging examinations have reliably contributed to early diagnosis of AD. Structural MRI is the most commonly used for diagnosis of AD because of its high spatial resolution and tissue contrast, while nonenhanced CT is generally not recommended since it is less sensitive than MRI in detecting AD. However, in some countries, the use of MR imaging is limited because of the high cost and the less availability. Due to these reasons, CT could be a favored method for identifying AD patients in many countries. Enlargement of the temporal horn of the lateral ventricle (THLV) on CT or MRI images is an accurate diagnostic marker of AD. The atrophy of the medial temporal lobe (MTL) is usually recognized in AD patients. The MTL atrophy causes the enlargement of the THLV adjacent to the MTL. Recently, we developed a z-score-based analysis of the THLV volume based on a voxel-by-voxel analysis on CT images. The previous study showed that regional mean z score of THLVs calculated using the method was highly correlated with the volume of the THLV for 50 subjects (R2 = 0.94). It demonstrated that the z-score-based method has the potential for quantitative estimation of the volume of the THLV on CT images [1]. Therefore, this method may be used for indirectly evaluating the atrophy of the MTL that results in diagnosing AD. In this study, we evaluated the performance of the z-score-based analysis of the THLV volume for detecting early AD on CT images. Methods The semi-quantitative analysis method consists of four steps, i.e., anatomic standardization, construction of a normal reference database, calculation of a z score, and calculation of the regional mean z score in a volume of interest (VOI). The regional mean z score was used as an index of THLV volume. First, for anatomical standardization, all CT data were transformed into a standard brain atlas by use of SPM8 software. The data were smoothed with a 4-mm full-width-at-half-maximum isotropic Gaussian kernel. Second, for the construction of a normal reference database, two control data sets (mean data and standard deviation data) were constructed by computing the average and the standard deviation of the CT value from the normalized CT database created from 40 normal subjects. Third, a z score was calculated for each voxel of an input data using the two control data sets. In the fourth step, two VOIs that covered the regions of the right and left THLV were created. Finally, the two VOI masks were applied automatically to the z-score dataset. Within each VOI mask, the regional mean z score was calculated. Fifty-one AD patients (mean age, 76.8 years) and 33 controls (mean age, 73.7 years) were used in this study. We selected retrospectively the 51 patients with a clinical diagnosis of AD according to the Diagnostic and Statistical Manual of Mental Disorders Fourth Edition (DSM-IV). These patients were classified into two groups of early and moderate-to-advanced AD. The former group and the latter group comprised 13 patients and 38 patients, respectively. The Mini Mental State Examination (MMSE) scores for the early AD group

Int J CARS ranged from 24 to 29 (mean, 26.1), and those for the moderate-toadvanced AD group ranged from 11 to 23 (mean, 18.2). We applied the z-score-based analysis to the above mentioned AD patients and the controls, and obtained 71 regional averaged z scores. Results Figure 1 shows box-and-whisker plots indicating the distribution of the regional averaged z scores of CT value in the VOIs for the early AD group, the moderate-to-advanced AD group and the control group. The median values of the z scores for the early AD patients, the moderate-to-advanced AD patients and the controls were 1.47, 1.86 and 1.07, respectively. The z score was significantly increased in the early AD patients compared with controls (P = .0017), and the z score was markedly increased in the moderate-to-advanced AD group compared with controls (P \ .0001).

Fig. 1 Box-and-whisker plot shows the distribution of averaged z score for the early AD group, the moderate-to-advanced AD group and the control group. The horizontal line in each box represents median value of the averaged z score. A circle indicates the outlier. A half of whole data lie in side of two boxes, and a total of 99 % of the data lie in side of the ranges of the whiskers The area under the ROC curve for distinction between the 51AD patients and the 33 controls was 0.826. Conclusion The results obtained from this study showed that the z-score-based analysis of the volume of the THLV has the potential to detect early AD on CT images. References [1] Takahashi N, Kinoshita T, Ohmura T, et al. (2016) Z-scorebased semi-quantitative analysis of the volume of the temporal horn of the lateral ventricle on brain CT images. Radiol Phys Technol, 9(1):69–76.

Information gain analysis of mammographic features of breast masses using machine learning R. Raju1 1 University of Illinois at Chicago College of Medicine, Chicago, United States

Keywords Information gain  Machine learning  Computer aided diagnosis  Mammography Purpose The purpose of this study was to use machine learning and information gain analysis as a means of determining the relative importance of diagnostic features of mammographic masses in determination of malignancy versus non malignancy. Methods Our study relied on a dataset created of 961 real cases of masses found on full field digital mammography that were pathologically confirmed as benign or malignant at Institute of Radiology of the University Erlangen-Nuremberg between 2003 and 2006 [1]. Each case that was used had 4 predictive attributes (margins of mass, shape of mass, radiodensity of mass, and patient age), an ordinal attribute comprised of the radiologist’s BiRads score of the lesion, and a binomial classification attribute (benign vs malignant). Of the 961 cases, 801 of them contained all 6 of the attributes required for our analysis. The BiRads category was removed from the dataset because of its dependence on information obtained from the other attributes in the dataset. Analysis of the data relied on Weka, an open source machine learning algorithm suite [2]. After the data was recompiled into a suitable format for Weka, the data was input into a supervised classifier algorithm to obtain a baseline classification performance with all 4 predictive attributes. Performance was measured using Correct Classification Rate, Area under the Reciever Operator Curve (AUC), and Kappa statistic. After this baseline was obtained, the attributes in the dataset were then ranked on the basis of information gain. Attributes were then systematically removed from the dataset, from the lowest information gain ranking to highest, and the classifier was reapplied to the lower dimensional dataset, with any change in classification performance being noted. Our choice of classifier was BaysNet, as it had the highest baseline performance for the mammography classification task. Stratified ten-fold cross validation was used to allow the use of one dataset for both training and testing of the classifier. Results For the task of classifying the cases as benign or malignant with the complete set of 4 predictive attributes using BaysNet, the correct classification rate was 80.6996 %, the AUC was .871, and the kappa value was .615. Information gain ranking for the attributes from highest information gain to lowest information gain was: Margin of Mass, Shape of Mass, Patient Age, and Density of Mass. When the classifier was applied to the 3 attribute modified dataset (Density of Mass removed), the correct classification rate was 80.4584 %, AUC = .859, and k = 6101. When applied to the 2 attribute modified dataset (Density of Mass and Patient Age attributes removed), the correct classification rate = 79.7346 %, AUC = .81, and j = .5952. For the data set with only Margin of Mass as a predictive attribute, the correct classification rate = 77.9252 %, AUC = .76, and k = .5615. Conclusion Of the four attributes of mammographic masses, the density of the mass appears to be least significant and the performance of the classifier does not appear to degrade significantly when this attribute is eliminated. As other attributes were removed from the dataset based on their information gain ranking, there was a predictable decrease in classifier performance. The highest ranked attribute, Margin of Mass, was by far the most informative attribute, as classifier performance was still robust with only that attribute present in the data set. References [1] Elter M, Schulz-Wendtland R, Wittenberg T (2007); The prediction of breast cancer biopsy outcomes using two CAD approaches that both emphasize an intelligible decision process. Medical Physics 34(11), pp. 4164–4172

123

Int J CARS [2]

Hall M, Eibe F, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009); The WEKA Data Mining Software: An Update; SIGKDD Explorations, Volume 11, Issue 1.

visual examination (Fig. 1c, d). The seven parameters are presented in Table 1. While all are informative, only two parameters (contrast and feature number) are discriminative i.e. significantly different between normal and cancer samples. Only contrast and feature number are significantly different than normal (*, p-value \ 0.05). The decision accuracy of the neural network was 84.5 %.

Automatic diagnosis module for in vivo optical biopsy L. Gruionu1, D. Stefanescu2, C. Streba2, T. Cartana2, A. Saftoiu2,3, G. Gruionu1,4 1 University of Craiova, Faculty of Mechanics, Craiova, Romania 2 University of Medicine and Pharmacy Craiova, Research Center of Gastroenterology and Hepatology, Craiova, Romania 3 Copenhagen University Hospital Herlev, Gastroenterology, Herlev, Denmark 4 Harvard Medical School, Department of Surgery, Boston, United States Keywords Computer aided diagnosis  CLE  Marching squares  Colon cancer Purpose Optical biopsy techniques have been developed to combine confocal microscopy with existing endoscopic equipment [1, 2] to reduce false negative biopsies in colorectal cancer (CRC). The aim of this study was to develop a computer aided diagnosis (CAD) algorithm for CRC, based on analyzing colon eCLE images, which can complement the existing immunohistological and imaging diagnosis methods. Methods This study was conducted on eCLE images from the database of the Research Center of Gastroenterology and Hepatology Craiova, Romania. A total number of 1035 images of normal or cancer colorectal mucosa (44.5 ± 21.3 and 75.4 ± 59.4 images per patient for normal and cancer respectively) were used for this analysis. Before the eCLE procedure, all patients signed an informed consent and the study was approved by the Committee of Ethics and Academic and Scientific Deontology from the University of Medicine and Pharmacy of Craiova. Colon endomicroscopy investigation was performed using the EC-3870 CIFK, colonoscope (Pentax, Tokyo, Japan). To obtain a histopathological diagnosis in real time, the examination was performed in vivo, after intravenous administration of 5 ml of 10 % fluorescein. eCLE scanning was performed on a region of interest identified during colonoscopy. The resulting grey-scale 1024 x1024 pixels images were acquired at a rate of 0.8 images/second and were stored for further analysis (200–300 images per examination). We processed the images using a computer aided diagnosis (CAD) module of a proprietary medical imaging system (NAVICAD). The module developed with the Matlab programming software (The MathWorks Inc. USA) computes the fractal dimension and lacunarity for every image, furthermore from the gray-level co-occurrence matrix (GLCM) the contrast (C), correlation (CO), energy (E) and homogeneity (H) measures are extracted. To identify specific anatomical features such as the glandular crypts, a Gaussian smoothing function is initially applied on images to reduce the noise. The contour of anatomical features are identified using a Marching Square and linear interpolation algorithm, by which the ratio aria/ perimeter is computed as a measure of feature roundness. The features (FN) with a ratio above an experimental determined value are counted for every image. A normal colorectal mucosa tissue has many close to circular features (ratio above 4) compared to pathological tissues (ratio below 2) with chaotic, tortuous structure. A two-layer feed forward neural network was used to diagnose images as normal or cancer based on the seven imaging parameters. Results eCLE imaging reveals the specific glandular crypts architecture of the normal mucosa (Fig. 1a, b). In contrast, the tumor mucosa shows disruption of the crypt hexagonal geometry which is hard to interpret by

123

Fig. 1 a. Normal colon mucosa with round shaped crypts (blue circle), dark goblet cells (yellow circles), and narrow and regular blood vessels surrounding the crypts (red arrows). b. Normal colon mucosa image processed: FD = 1.732, L = 0.13, C = 0.26, CO = 0.97, E = 0.24; H = 0.89; FN = 14. c. Adenocarcinoma with disorganized mucosa and lack of structure, crypts are elongated with irregularly thickened epithelium (blue), dilated and distorted blood vessels (red arrows). d. Adenocarcinoma image processed: FD = 1.67; L = 0.03; C = 0.10; CO = 0.98; E = 0.15; H = 0.95; FN = 0

Table 1 Average CAD parameter values ± standard deviation Normal

Cancer

Fractal dimension

1.88 ± 0.05

1.91 ± 0.04

Lacunarity

0.05 ± 0.02

0.04 ± 0.02

Contrast

0.28 ± 0.04

0.24 ± 0.03*

Correlation

0.93 ± 0.02

0.93 ± 0.03

Energy

0.14 ± 0.02

0.17 ± 0.04

Homogeneity Feature no

0.87 ± 0.01 4.7 ± 1.98

0.88 ± 0.01 1.52 ± 0.71*

Conclusion CLE has an increased accuracy for the in vivo histological diagnosis of colorectal cancer [1, 2], based on real time interpretation of glandular crypts architecture. To improve the diagnosis accuracy we developed an objective method of interpretation based on several anatomical features, with a decision accuracy of 84.5 %. A set of seven parameters was obtain to identify those relevant features that can distinguish classes of analyzed lesions and we used a neural network analysis to study the combined effect of all parameters. Acknowledgements The research leading to these results has received funding from EEA Financial Mechanism 2009—2014 under the project EEA-JRP-RONO-2013-1-0123—Navigation System for Confocal Laser Endomicroscopy to Improve Optical Biopsy of Peripheral Lesions in the Lungs (NAVICAD), contract no. 3SEE/30.06.2014. References [1] Gheonea DI, Caˆrt¸ aˆnaˇ T, Ciurea T, Popescu C, Baˇdaˇraˇu A, Saˇftoiu A. Confocal laser endomicroscopy and immunoendoscopy for real-time assessment of vascularization in gastrointestinal malignancies. World Journal of Gastroenterology 2011; 17(1):21–27. [2] Yousuke N, Hiroyuki I, Susumu S, Takuji I, Jason B. S, Kenneth J. C, et al. Confocal laser endomicroscopy in gastrointestinal and pancreatobiliary diseases. Digestive Endoscopy 2014; 26 (Suppl. 1): 86–94.

Int J CARS

A method for highlighting lung tuberculosis lesions in CT images using superpixel approach V. Liauchuk1, V. Kovalev1, A. Astrauko2, A. Rosenthal3, A. Gabrielian3 1 United Institute of Informatics, Biomedical Image Analysis, Minsk, Belarus 2 Scientific and Practical Center for Pulmonology and Tuberculosis, Minsk, Belarus 3 National Institute of Allergy and Infectional Diseases, Bethesda, United States Keywords CT  Tuberculosis  Superpixel  Co-occurrence Purpose An automatic highlighting of lung tuberculosis (TB) lesions in CT images is one of the important problems in corresponding CAD systems, PACS environment and thematic web-portals [1]. Recently several methods are suggested for this problem. The most popular of them are based on image description approach suggested in [2] which considers local histograms of a collection of filtered versions of the image. The purpose of this study is to present a method for highlighting TB lesions which utilizes an approach for image description based on superpixels [3]. The proposed image description approach has reasonable performance and allows to project feature vector elements all the way back onto the original image for more accurate highlighting [4]. Methods Materials. For training and testing image description methods we used a dataset of 270 CT slice regions (178 normal tissue samples, 92 TB lesion samples) of 128 9 128 pixels in size extracted from CT scans of 120 TB patients. The developed highlighting method was finally tested on 3D CT images. Methods. The proposed highlighting method is based on an image description approach which considers co-occurrence of superpixel classes. The procedure consists of four major stages. Stage-1: Creating a superpixel dictionary. The image dataset is used to retrieve a collection of superpixels extracted from each image. For every superpixel, 4 intensity (mean, STD and entropy of intensity, mean gradient) and 2 shape features (compactness and ‘‘squareness’’) were calculated. Finally, k-means clustering algorithm was applied to the extracted feature vectors with number of clusters being set to N. The resultant cluster centroids represent the corresponding superpixel dictionary. Stage-2: Image description. For each sample image superpixels are extracted and categorized into N classes according to the pre-calculated dictionary. Finally, N 9 N co-occurrence matrices of adjacent superpixel classes are used as image descriptors. Stage-3: Pattern recognition. This stage includes applying Principal Component Analysis (PCA) to the table of descriptors followed by classification. Finally, for each descriptor element (i.e. pair of adjacent superpixels) a lesion-likelihood score is calculated with the use of coefficients provided by Logistic Regression classifier. Stage-4: Highlighting. This stage considers building a ‘‘heatmap’’ by means of summarizing the corresponding superpixel scores obtained at the previous stage all over the target image. Results Superpixels were generated using all possible combinations of the following parameter values: Size = 8, 16, 32; Regularization = 0.3, 0.1, 0.03 and 0.01 (see examples on Fig. 1). The dictionary size N was set to 8, 16, 32, 64 and 128. For assessing the accuracy of recognition of lesion images the 5-nearest neighbors method was used within the v-fold cross-validation scheme (v = 5).

Fig. 1 Examples of normal and TB lesion images and the corresponding image superpixels Left panel of Fig. 2 illustrates the performance of the superpixelbased image description method (Size = 16, Regularization = 0.3, N = 8) being compared to other methods. Right panel of Fig. 2 demonstrates example CT slices with TB-relevant structures highlighted using the proposed method.

Fig. 2 Classification accuracy produced by different image description methods (SP—superpixel-based, ImFilt—image filtration, CM— gray-level co-occurrence matrices, LBP—local binary patterns) and example CT slices with highlighted TB-relevant structures Conclusion The proposed image description technique may provide performance well comparable to the one of conventional image description methods. Also the technique provides ability for back projection onto the original image. Results of our study suggest that the proposed method can potentially be employed as a tool for highlighting lung tuberculosis lesions in CT images. Such tool can be useful for detection and visualization of representative slices on 3D CT images (see demo version at [5]). References [1] http://tuberculosis.by/. Last visited 01.02.2016. [2] van Ginneken B, ter Haar Romeny BM (2003) Multi-scale texture classification from generalized locally orderless images. Pattern Recognition, 36(4):899–911. [3] Achanta R, et al. (2012) SLIC Superpixels Compared to Stateof-the-art Superpixel Methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11):2274–2282.

123

Int J CARS [4]

[5]

Kovalev VA, et al. (2011) A method for identification and visualization of histological image structures relevant to the cancer patient conditions. Proc. of CAIP-2011, Spain, 6854(1):460–468. http://imlab.grid.by/appctslice/- last visited 01.02.2016.

Value of nested contours analysis algorithm in mammographic image processing I. Egoshin1, D. Pasynkov2, A. Kolchev3, I. Kliouchkin4 1 Mari State University, Yoshkar-Ola, Russian Federation 2 Oncology Dispenser of Mari-El Republic, Yoshkar-Ola, Russian Federation 3 Kazan Federal University, Kazan, Russian Federation 4 Kazan State Medical University, Kazan, Russian Federation Keywords Mammogram  Image processing  Contours analysis  Malignant masses Purpose Mass is the most commonly seen lesion on mammograms, that may correspond to both benign and malignant breast tumors. Despite typical characteristics of benign and malignant masses vary significantly, there is a wide grey area, that includes lesions of both types. This problem is especially important for small lesions, which may have very similar appearance. Therefore the first step of the CAD system is the identification of suspicious masses. Our aim was to develop the nested contours analysis algorithm (NCAA) and test its suitability for mammographic image processing. Methods NCAA involved the form and center position similarity assessment of contours of areas, which have the same range of brightness (according to a 256-gradations grey scale). The search of these areas started with maximal brightness on the source mammographic image. During this operation we used the fixed step. The operation continued until the minimum significant brightness threshold. During the each iteration we discarded the areas of image, whose brightness was lower than current gradation and defined the contours of all remaining objects. For each contour we assessed the following parameters: length, centre of gravity, entropy, eccentricity, compactness, moments of different degrees etc. Then we compared the contours of current

123

brightness gradation with similar nested contours of next gradation. In case they were different we concluded that this contour doesn’t represent a mass and should be discarded. The criteria of contours similarity were developed based on the training set that had no visible masses on the images. The more similar nested contours we found, the higher probability of mass we finally concluded. For testing we used mammograms of 109 women with morphologically verified breast cancer. Of them 23 women had type A density according to the American College of Radiology (2013) classification, 34—type B and 52—type C. Totally we tested 396 images, 190 of them contained 206 confirmed malignant masses, remaining didn’t have masses. The mean size was 15 ± 8 mm (range: 7–32 mm). Results The overall sensitivity of our algorithm for ACR A breasts was 77.3 % (34 of 44 masses), specificity—71.4 %, positive predictive value (PPV)—77.3 %, negative predictive value (NPV)—75.0 %. It should be noted that the majority (35/44) of lesions on ACR A images were small enough (mean size—9 ± 4 mm), that significantly improve their prognosis. For ACR B breasts the results were relatively similar: sensitivity was 81.1 % (30 of 37 masses), specificity—68.1 %, PPV—58.8 %, NPV—65.8 %. At the same time the size of this group lesions was slightly higher (mean—17 ± 6 mm), than in ACR A group. For ACR C breasts the sensitivity was predictably lower—62.4 % (78 of 125 masses), specificity—55.6 %, PPV—56.3 %, NPV— 62.7 %. The algorithm usually was unable to detect masses poorly visible or invisible on standard images, regardless of their size, as well as microcalcification clusters. The average rate of false-positive markings was one mark on every 5 images—for ACR A group, 0.5 marks per image—for ACR B group and 1.6 marks per image for ACR C group. At the same time, the majority (67.3 %) of these false positive marks were represented by fibrous band-shaped structures that can be easily filtered during the future processing. Conclusion NCAA can be used as a first-step mass detection method, however it needs additional filters for false positive band-like structures removal and probably for asymmetric areas transformation to a mass-like structures.

Int J CARS

Poster Session 22nd Computed Maxillofacial Imaging Congress

123

Int J CARS An augmented reality guidance system for orthognathic surgery using a tablet PC S.- J. Lee1, S.- Y. Woo1, S.- R. Kang1, W.- J. Lee2, J.- Y. Yoo1, W.- J. Yi3 1 Seoul National University, 1. Department of Biomedical Radiation Sciences, GSCST, Seoul, South Korea 2 Seoul National University, 2. Interdisciplinary Program in Radiation Applied Life Science, College of Medicine, Seoul, South Korea 3 Seoul National University, 3. Department of Oral and Maxillofacial Radiology, School of Dentistry, Seoul, South Korea Keywords Augmented reality  Image-guided surgery  Tablet PC  Orthognathic surgery Purpose An orthognathic surgery is aimed at correction of dento-facial defects. Correcting the maxillofacial deformities was performed by repositioning bone segments to an appropriate location according to the preoperative planning. Several image-guided orthognathic surgery system using virtual reality have been developed and applied to the patients. Their systems used various tracking systems, such as an infrared optical tracking system, electromagnetic tracking system and a system using streoscopic cameras, to navigate jaw segment and visualized virtual reality images during surgery [1, 2, 3]. However, the conventional systems have a several limitations. First, the systems didn’t provide a genuine camera image to the surgeon, which caused misunderstanding and losing a concentration on surgical sites and processes. Because the surgeons dispersed their attention to surgical components, such as a virtual reality screen, an actual surgical site and their surgical instruments [1, 2]. Second, intraoperative registration between patient and image has increased the complexity of surgical procedures and the operative time and labor within the surgical navigation system [3]. To address this issue, we developed an image-guided orthognathic surgery system adding an augmented reality using a tablet PC. Methods A. System overview—The system was consisted of a workstation, tablet-PC (Sony, Seoul, Korea), and the optical tracking system (NDI Spectra, Waterloo, Canada) (Fig. 1). Serial communication and TCP/ IP communication were used to deliver tracking information between each device.

Fig. 1 Overview of the present system B. CT scanning and model—The CT image was obtained using a MDCT (Siemens Medical Solutions, Munich, Germany) [2]. The 3D model structures were produced by a marching cube algorithm using 3D slicer, a free software for biomedical research. C. Preoperative Image registration—The process was divided into two independent steps. Matching the patient’s world coordinate with

123

the CT image coordinate was the first step. Another step is to match the CT image coordinate with the augmented reality (AR) image coordinate. A Point-to-point registration method was used in both processes [2]. D. Augmented reality guidance—The system provide two separated modes. The first one was a default mode that bone segment model image was superimposed on the patient’s face and tracked. The other mode was an osteotomy line guide mode. This mode augmented the cutting line on the surface of the patient’s face. For tracking, a reference tracking tool was attached on a forehead of the patient and a tablet tracking tool and maxillary tracking tool were attached on the tablet and a patient’s splint, respectively. During the operation, movement of the patient and the tracking tools were continuously tracked and calculated in real time. The movement of the augmented model was calculated according to the equation, represented in Fig. 2

Fig. 2 Equation for model visualization and AR guidance. TModel: current position model, MCT_NDI: registration matrix between CT and OTS, MCT_AR: registration matrix between CT and AR, Tinit_Tab and Tinit_Max: initial positions of the tablet and maxillary tracking tool, and TCurr_Tab and TCurr_Max: current positions of the tablet and maxillary tracking tool E. Accuracy Evaluation—To quantify accuracy, we calculated the root mean square (RMS) and the absolute difference between the landmark positions on the CT image and the world coordinates. For calculating, we performed a simple matrix calculation process to coincide the coordinate by multiplying the homogeneous transform matrix between two coordinates by the landmarks positions. Results The generated 3D model was transferred and directly superimposed on the skull phantom model on the simulation board in the laboratory through AR technology (Fig. 2). After the preoperative registration processes, the model was handled freely by the operator. During realtime guidance, the 3D augmented model was visualized in a red color with transparency. We also superimposed the cutting line model on the surgical site to provide the accurate perception of osteotomy line for surgeons (Fig. 2). For the accuracy evaluation of the system, the accuracy was calculated by comparing the landmark positions in the CT coordinate and world coordinate system using a simple matrix calculation process. For eight landmarks on the skull phantom model, the mean absolute differences were 1.50 ± 1.25, 1.42 ± 1.44 and 1.74 ± 1.27 mm on the x-, y-, and z-axes, respectively, and the mean RMS difference was 3.27 ± 1.17 mm (Tbl. 1). Conclusion In this study, we have developed an augmented reality guidance system for orthognathic surgery using a tablet PC. The preoperative registration processes reduced the complexity of surgical procedures and the operative time and labor for both surgeons and engineers. Superimposing of the 3D anatomical model on the surgical site increased comprehension of the geometrical information of the surgical sites and enhanced concentration on the surgery processes for surgeons. In future studies, we will apply this augmented reality guidance system on the patients for orthognathic surgery in operating room (Table 1).

Int J CARS Table 1 Absolute difference in x-, y-, and z-axes, and root mean square (RMS) difference between the landmark positions Landmark number

Axis (mm) x

y

z

1

0.38

2.59

1.87

2

0.92

1.48

0.47

3 4

3.29 0.53

0.02 4.34

1.97 1.15

5

1.77

1.17

2.25

6

0.33

1.03

0.91

7

3.42

0.64

0.82

8

1.34

0.11

4.48

Mean

1.50 ± 1.25

1.42 ± 1.44

1.74 ± 1.27

RMS

3.27 ± 1.17

Acknowledgement This study was supported by a grant of the Korean Health Technology R&D Project, Ministry of Health & Welfare, Republic of Korea (HI13C1491). References [1] Bobek SL: Applications of Navigation for Orthognathic Surgery. Oral Maxil Surg Clin 26:587–?, 2014. [2] Kim DS, et al., ‘‘An integrated orthognathic surgery system for virtual planning and image-guided transfer without intermediate splint’’ Journal of Cranio-Maxillo-Facial Surgery, vol. 42, pp. 2010–2017, 2014 [3] Li B, Zhang L, Sun H, Shen SGF, Wang XD: A New Method of Surgical Navigation for Orthognathic Surgery: Optical Tracking Guided Free-Hand Repositioning of the Maxillomandibular Complex. J Craniofac Surg 25:406–11, 2014.

The location of the mandibular canal in the posterior mandible of the jaw deformity patients: observed with pre-surgical MDCT images T. Kawai1, Y. Kumazawa1, R. Asaumi1, M. Mizutani1, A. Yamaguchi1, T. Yosue1 1 Nippon Dental University, Oral and Maxillofacial Radiology, Tokyo, Japan Keywords CT  Jaw deformity  Mandible  Mandibular canal Purpose The knowledge of the location and the course of the mandibular canal is an important for surgical procedure to the mandible, such as placement of dental implants or orthognathic surgery. The sagittal split rams osteotomy (SSRO) is the common surgical procedure for the jaw deformity patients although, post operative complications were reported. One of the most frequent post-operative complications is neurosensory disturbance owing to the injury to the mandibular canal which contains inferior alveolar neuro-vascular bundle. The purpose of this study was to evaluate the course of the mandibular canal in the posterior mandible of the jaw deformity patients (Angle class 3 malocclusion) of Japanese, using pre-operative multi detector CT (MDCT). Methods For the analysis, the MDCT images of the 20 jaw deformity patients acquired for the preoperative planning of the SSRO surgery were

used. All 20 patients (male: 8, female: 12) were diagnosed as the Angle class 3 malocclusion in the other clinical examinations. MDCT images of all patients were taken by the Aquilion super4 (Toshiba Medical Systems Co, Tochigi, Japan) with the tube potential of 120 kV, slice thickness of 0.5 mm or 1.0 mm, and FOV of 24 cm. After acquisition of MDCT data, all data were transferred to a workstation and evaluated using zioTerm 2009 software (Ziosoft, Inc, Tokyo, Japan). The standard plane of all MDCT data was set at occulusal plane. For the analysis of the location of the mandibular canal in the posterior mandible, the bucco-lingual width of the mandible through the center of the mandibular canal and bucco-lingual location of the mandibular canal were measured at the center of the first molar, and the distal edge of the second molar on the cross sectional images. The bucco-lingual locations of the mandibular canal at ramus region were observed on the axial images parallel to the occulusal plane. The mandibles which had irregular trabecular bone (could not observed mandibular canal), and which had been extracted premolar during presurgical orthodontic treatment were excluded from this study. Results In 32 sides of 16 mandibles (80.0 %) represented the course of the mandibular canal clearly. The average bucco-lingual width of the mandible at center of the first molar and the distal edge of second molar was 12.0 mm, and 13.3 mm respectively. The average distance between mandibular canal lingual/buccal outer cortical and the edge of the lingual/buccal cortical bone were 2.9 mm/6.3 mm and 3.6 mm/ 6.7 mm respectively. In 31 of 32 sides at the center of the first molar and 26 side at the distal edge of the second molar, the center the mandibular canal was located lingual side on the cross sectional images. In the ramus region, 6 of the 32 sides (18.8 %) in 3 mandibles (18.8 %), mandibular canal had contact with inner surface of the buccal cortical bone (no trabecular space between them). Conclusion Detailed understanding of the location and the course of the mandibular canal is critical to the surgical procedures. When performing SSRO, the surgical risk would be increase in cases of the mandible which has thin bucco-lingual bone width and narrow buccal trabecular space. In this study, it was confirmed in three mandibles there was no trabecular space between the mandibular canal and the buccal cortical bone. To reduce the injury to the inferior alveolar neurovascular bundle and consecutive postoperative neuro-sensory disturbance, the clinician should aware of the anatomical status of the mandible by three dimensional image modality such as MDCT or CBCT.

Comparison of dual-energy and single-energy CBCTs for assessment of trabecular bone microarchitecture S.- R. Kang1, S.- J. Lee1, S.- Y. Woo1, W.- J. Lee2, W.- J. Yi1,3 1 Seoul National University, Department of Biomedical Radiation Sciences, Graduate School of Convergence Science and Technology, Seoul, South Korea 2 Seoul National University, Interdisciplinary Program in Radiation Applied Life Science, College of Medicine, Seoul, South Korea 3 Seoul National University, Department of Oral and Maxillofacial Radiology, School of Dentistry, Seoul, South Korea Keywords Trabecular bone  Bone micro-architecture  Dual-energy CBCT  Micro-CT Purpose The quality of bone, as well as its quantity, is one of the essential factors in diagnosing diseases associated with bone loss and predicting the success rate of implant installation [1]. Bone quality is considered a consolidation of bone mass, material properties, and

123

Int J CARS structural properties [1]. The structural properties of bone include its microarchitecture as well as its geometry [1]. Therefore, evaluation of trabecular microarchitecture can provide effective information for implant installation or diagnosis of bone pathology. Recently, CBCT has been widely used in dental clinic such as diagnosis of maxillofacial disease and treatment planning for dental implants because of its less cost for scanning and low radiation exposure [2]. Therefore, CBCT is suggested as an imaging modality to analyze the bone microarchitecture parameters. However, most studies using CBCT were limited to few specific parameters such as BV/TV and Tb.Th for assessment of bone quality. Lately, there has been a rapid increase in the use of dual-energy CT. The DECT system uses two different energies simultaneously for the imaging. Theoretically, if the low and high voltage images are adequately fused, we can obtain the images that balance the advantages and disadvantages of both low and high voltage images according to the attenuation difference between the lesion or structures of main interest and the background material [3]. We used the DECT method, which is linearly combining the low- and high-energy images, in CBCT and analyzed the trabecular bone parameters with micro-CT data. The purpose of this study was to compare the correlations of microarchitectural parameters of trabecular bone by dual-energy or singleenergy CBCTs with by micro-CT. Methods A total of 30 dental implants (11.5 mm length and 4.5 mm diameter) were placed into swine bone specimens. The bone samples were trimmed to a cube of uniform size, 20*20*20 mm3, for micro-CT scanning and stability measurement. The micro-CT examination of each bone specimen was conducted using a micro-CT. The scanning parameters were set at 100 kVp, 0.1 mA, 0.59 s, and a rotation step of 0.2 during 360 rotation, and an isotropic voxel size of 12.97 lm. CBCT scanning was performed with a bench-top CBCT system that we set up. Dual-energy CBCT images of bone samples were obtained at 40 and 80 kVp with an isotropic voxel size of 129 lm, while single-energy CBCT images were acquired at only 80 kVp tube voltage. The volume image was reconstructed by using the filtered back projection algorithm with a ramp filter. The dual-energy CBCT images were weighted and combined by optimal linear energy weighting method [3, 4]. Each energy bin data are fused using correspondent weighting factors and it can make an increase of the images’ CNR [3, 4]. The micro-CT and CBCT images were segmented to quantify the trabecular bone micro-architecture using the CTAn software. The following 3D microstructural parameters were calculated for each sample: percent bone volume (BV/TV), bone specific surface (BS/ BV), trabecular thickness (Tb.Th), trabecular separation (Tb.Sp), trabecular number (Tb.N), trabecular bone pattern factor (Tb.Pf), structural model index (SMI), and fractal dimension (FD). The relationships between bone microstructural parameters from dual-energy CBCT and micro-CT data and between the parameters from single-energy CBCT and micro-CT images were evaluated using Pearson’s correlation analysis and linear regression analysis by SPSS 21. Results Table 1 shows the Pearson’s correlation coefficients between microarchitecture parameters measured by single-energy or dual-energy CBCTs with micro-CT. Most of the microstructural parameters from dual-energy CBCT images showed significantly greater correlation with parameters from micro-CT images than from singleenergy CBCT. Especially, BS/BV, Tb.Pf and Tb.Sp showed the highest increment of correlation coefficients. All parameters showed the linear relationships with parameters from micro-CT images (Fig. 1). Among the parameters, the BV/TV, BS/BV, SMI, Tb.Pf, Tb.Th, Tb.N, and FD from single-energy CBCT showed strong correlations (P \ 0.01)

123

Table 1 Pearson’s correlation coefficients between microstructure parameters measured by single-energy or dual-energy CBCTs with by micro-CT, and the values for the percentage increase comparing single-energy and dual-energy CBCTs Micro-CT vs CBCT_S

Micro-CT vs CBCT_D

Increase

BV/TV

0.857

0.903

5.37 %

BS/BV

0.533

0.581

9.01 %

SMI

0.666

0.704

5.71 %

Tb.Pf

0.524

0.622

18.70 %

Tb.Th

0.683

0.709

3.81 %

Tb.N

0.647

0.695

7.42 %

Tb.Sp

0.495

0.515

4.06 %

FD

0.539

0.566

5.01 %

Fig. 1 Relationships between 3D bone microstructural parameters of BV/TV (a), BS/BV (b), SMI (c), Tb.Pf (d), Tb.Th (e), Tb.N (f), Tb.Sp (g), and FD (h) from the micro-CT and dual-energy CBCT (p) Conclusion 3D trabecular microstructural parameters from dual-energy CBCT images had stronger correlation with those from micro-CT images, compared to single-energy CBCT images. The results suggest that the method of dual-energy CBCT offers more reliable evaluation of trabecular bone microarchitecture then single-energy CBCT imaging, especially for the parameters related to connectivity and configuration of trabeculae. Acknowledgements This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MEST) (No. 2013R1A2A2A03067942), South Korea. References [1] Bouxsein ML (2003) Bone quality: where do we go from here? Osteoporosis international : a journal established as result of cooperation between the European Foundation for Osteoporosis and the National Osteoporosis Foundation of the USA 14 (Suppl 5):S118–127.

Int J CARS [2]

[3] [4]

Kim JE, Yi WJ, Heo MS, Lee SS, Choi SC, Huh KH (2015) Three-dimensional evaluation of human jaw bone microarchitecture: correlation between the microarchitectural parameters of cone beam computed tomography and micro-computer tomography. Oral surgery, oral medicine, oral pathology and oral radiology 120:762–770. Schmidt TG (2009) Optimal ‘‘image-based’’ weighting for energy-resolved CT. Medical physics 36: 3018–3027. Schmidt TG (2010) CT energy weighting in the presence of scatter and limited energy resolution. Medical physics 37:1056–1067.

Maxillary surgical template for repositioning of maxillomandibular complex without using an intermediate splint in orthognathic surgery J. B. Park1,2, S. J. Hwang1,2,3, J. J. Han1,4 1 School of Dentistry, Seoul National University, Department of Oral and Maxillofacial Surgery, Seoul, South Korea 2 Seoul National University Dental Hospital, Department of Oral and Maxillofacial Surgery, Seoul, South Korea 3 Seoul National University Dental Hospital, Orthognathic Surgery Center, Seoul, South Korea 4 Chonnam National University Dental Hospital, Department of Oral and Maxillofacial Surgery, Gwangju, South Korea Keywords Orthognathic surgery  Surgical template  Computeraided design (CAD)  Computer-aided manufacturing Purpose Accurate repositioning of the maxillary and mandibular segment is essential to improve esthetics and function in orthognathic surgery. With the improvement of three-dimensional (3D) imaging technology and computer-aided design and manufacturing techniques (CAD/CAM), various CAD/CAM templates have been developed as alternatives to the traditional error-prone and time-consuming intermediate splint. However, the majority of previously developed templates still used an intermediate splint and transferred the preoperative virtual plan to the real operation field indirectly. Here, we introduce a technical note regarding maxillary surgical templates consisting of osteotomy and repositioning guide templates. Methods Maxillary surgical templates consisted of two templates: the osteotomy guide template and the repositioning guide template. The osteotomy guide template included two parts (upper and lower) that were connected by two bridges, which was designed to cover the maxillary anterior wall for exact and passive positioning. The osteotomy guide template provided information about the osteotomy line and bony interference. The upper part of the osteotomy guide template had a wedge-shaped female-structure, which should be joined with the wedge-shaped male-structure of the repositioning guide template during the surgery. After removal of the lower part of the osteotomy guide template, the repositioning guide template was placed on the right and left side of the maxillary segment as a second template. This template contained drill holes, which were located at the same positions as those in the lower part of the osteotomy guide template. Correct vertical and horizontal positioning of the maxilla in relation to maxillary canting and midline correction, and mediolateral and superoinferior movement can be easily examined intraoperatively by adjusting the wedge-shaped male and female structures on both sides. Correct anteroposterior maxillary positioning can also be easily confirmed by the observation of an even anterior surface of the

wedge-shaped male and female structures without any step on both sides. Results A patient with mandibular prognathism and facial asymmetry took preoperative 3D facial CT. The CT images were subsequently converted into Stereolithography Interface Format (STL) files using Mimics Software 17.0 (Materialise, Leuven, Belgium). The maxillary surgical templates for the orthognathic surgery were designed and fabricated after virtual osteotomy and simulation surgery. In the real surgery, the maxillary surgical templates were applied in a patient with mandibular prognathism and facial asymmetry, After Le Fort I oseotomy of the maxilla and bilateral sagittal split ramus osteotomy of the mandible, the maxillomandibular complex fixed with final occlusal splint was repositioned simultaneously using this templates system. The accuracy of repositioning of the maxillomandibular complex was confirmed using intraoperative navigation system. Conclusion The guiding functions of CAD/CAM devices for maxillary positioning without the use of an intermediate splint are based on the adaptation of templates to anatomical structures solely at the anterior maxillary surface, and these devices do not guide the position of the posterior maxilla. A small amount of clockwise or counterclockwise pitching rotation can occur, and this pitching rotation can lead to the pogonion being greatly protruded or retruded. Therefore, it is necessary to control maxillary positioning precisely, especially in ‘‘pitch’’ movement with intraoperative navigation system. Using these templates, the maxillomandibular complex can be successfully repositioned without using an intermediate splint.

Anatomical structure analysis of dental caries using optical coherence tomography for medical imaging R. E. Wijesinghe1, N. H. Cho1, K. Park1, M. F. Shirazi1, M. Jeon1, J. Kim1 1 Kyungpook National University, Electronics Engineering, Daegu, South Korea Keywords SD-OCT  Dentail caries  Enamel demineralization  Cavity Purpose Optical coherence tomography (OCT) has proven to be able to provide cross-sectional (2D) and volumetric (3D) images of scattering biological tissues for medical diagnostics. The main aim of this study was to analyze dental caries and cavity regions quantitatively, and to examine the micro-anatomical structures of the enamel residual more precisely. We demonstrated the rapid diagnostic ability by using a high-speed, 1310-nm spectral domain optical coherence tomography (SD-OCT) system to acquire 2D and 3D images of demineralized carious ex vivo posterior teeth samples [1, 2]. An accurate volume, thickness and depth of caries were evaluated through the quantitative analysis. These results show that OCT with the proposed method can systematically inspect the demineralization and it can play an important role in prevention. Methods The image acquisition of the experiment was carried out using 1310 nm spectral domain OCT (SD-OCT) system. Figure 1 shows the schematic diagram of the system configuration. The broadband light source of the OCT system is a superluminescent diode (SLED, Denselight Semiconductors) with a bandwidth of 135 nm. A 50:50 optical fiber coupler was used to split the broadband light beam into the sample and reference arms. The backscattered light beam was propagated towards the detector. A 14-bit complementary metal-

123

Int J CARS oxide semiconductor (CMOS) line scan camera (SU-1024LDM Compact; Goodrich USA 1024 pixels) was used as the detector. The SNR of the system was 110 dB and the axial resolution was measured as 5.6 lm. Ex vivo (canine, molar and pre-molar) carious tooth samples were used to obtain 2D and 3D images.

using a well-known medical imaging modality called optical coherence tomography system. The obtained cross-sectional 2D images, 3D volumetric images, and the quantitative results revealed the capability of our system to detect dental caries accurately compared to the existing medical diagnostic methods, which will be an important fact for the detection of caries. Therefore, the physicians are able to diagnose the tooth volumetric and thickness changes, which will be useful to barricade the progression of caries at an advanced stage. References [1] Lee RC, Kang H, Darling CL, Fried D: Automated assessment of the remineralization of artificial enamel lesions with polarization-sensitive optical coherence tomography. Biomedical optics express 5(9), 2950–2962 (2014). [2] Bu¨hler C, Ngaotheppitak P, Fried D: Imaging of occlusal dental caries (decay) with near-IR light at 1310-nm. Opt. Express 13(2), 573–582 (2005).

Accuracy of reorientation methods of 3-dimensional surface skull model as natural head position Fig. 1 Schematic diagram of the spectral-domain optical coherence tomography (SD-OCT) system. Note the use of the following acronyms in the figure: BLS: broadband laser source, C: collimator, DG: diffraction grating, FC: fiber coupler, GS: galvano scanner, L: lens, CMOS: Line scan camera, M: mirror, PC: polarization controller Results The obtained 2D images provided a clear visualization of the carious regions including tooth cavities, fissures, and pits regions. The infected micro-structures as well as the enamel loss were clearly observed through the obtained 3D volumetric images, and also they provide accurate anatomical information about the carious regions owing to the high depth penetration. OCT signal based depth profile analysis method was demonstrated to quantitatively evaluate the depth and the thickness of the cavities, dental caries and the demineralized enamel structures. The obtained 2D image, depth profile analysis plot, and 3D image are shown in Fig. 2. The 2D (Fig. 2(a)) and 3D (Fig. 2(c)) images represent the carious region of the tooth sample providing a clear visualization of the cavity and fissure regions. The depth profile is shown in Fig. 2 (b). The red color dashed graph of the depth profile depicts the carious region and the white color dashed graph depicts the healthy region. The precise thickness of the carious regions and the enamel residual could be numerically evaluated in micrometer range through the depth profile analysis. Thus, the obtained data will be useful for the detection of caries because the gradual volume reduction of the enamel due to the gradual growth of caries can be detected quantitatively in advance.

Fig. 2 (a) 2D OCT image. (b) Depth profile analysis plot of 2(a). (c) Top view of the 3D image of the carious tooth sample Conclusion We demonstrated an initial ex vivo study to identify the internal microstructures of dental caries and to quantify the enamel residual

123

H. J. Yang1, S. J. Hwang1,2,3, J. J. Han4 1 Seoul National University Dental Hospital, Orthognathic Surgery Center, Seoul, South Korea 2 School of Dentistry, Seoul National University, Department of Oral and Maxillofacial Surgery, Seoul, South Korea 3 Seoul National University Dental Hospital, Department of Oral and Maxillofacial Surgery, Seoul, South Korea 4 Chonnam National University Hospital, Department of Oral and Maxillofacial Surgery, Gwangju, South Korea Keywords Natural head position  POSIT algorithm  Orthognathic surgery  3D CT Purpose Natural head position (NHP) is a stable physiologic position ‘‘when a man is standing and his visual axis is horizontal’’. An accurately recorded NHP is vital for clinicians in the diagnosis and treatment of patients with craniomaxillofacial deformities, especially for for patients with significant facial asymmetries. Three-dimensional (3D) virtual surgery using computed tomography (CT) is commonly used in the planning of orthognathic surgery. However, a general CT image does not reflect the patient‘s NHP, because the patient is in the supine position with their head randomly oriented during a medical CT scan. Therefore, it is important to correct the head position of 3D model before virtual surgery. The purpose of this study was to analyze the effects of the application of NHP correction using pose from orthography and scaling with iteration (POSIT) algorithm in 3D virtual surgery, and to compare NHP correction using POSIT algorithm and manual re-orientation using clinical photographs. Methods 12 patients took the frontal and lateral photos in the NHP with attachment of the five ceramic markers on their faces, then, CT was performed. The head position of 3D models was corrected by four different ways; (1) NHP correction using POSIT algorithm; (2) rotation of 3D models on the basis of ceramic markers on the photos taken with the NHP; (3) rotation of 3D models on the basis of anatomical structure on the photos taken with the NHP; (4) random adjustment to fit the anatomical symmetry. Since then, 3D virtual surgery was performed in the same way as a real surgical plan. The vertical, mediolateral, anteroposterior positions of maxilla and mandible relative to the plane passing through the Na were analyzed three-dimensionally. Data was statistically analyzed. Results The head position correction methods, (2, 3, 4) showed statistically significant differences in the anteroposterior position of maxilla and

Int J CARS mandible, midline and chin deviation, maxillary cant and yaw compared to the head position by NHP correction using POSIT algorithm. For the head position correction methods (2, 3, 4), mean correction errors in the maxillary midline deviation and chin deviation were about 1.12 mm (0.02 * 3.70 mm) and 1.70 mm (0.04 * 5.25 mm), respectively. Mean correction errors in the maxillary cant and yaw 0.52 mm (0.00 * 3.41 mm) and 0.53 (0.27 * 1.86), respectively. The correction errors in the anteroposterior position of maxilla and mandible were significantly greater in random adjustment method (4) compared to NHP correction methods using anatomical structure (3). Conclusion The exact correction of NHP should precede 3D virtual surgery, because incorrect head position results in a difficulty in determining the amount of surgical correction. The head position of 3D model can be corrected easily with the NHP correction method using the clinical photo taken in the NHP. However, there could be great error during NHP correction using the clinical photo due to the difficulty of manual repositioning or the ambiguity of the anatomical structure in some patients. The head position of 3D model can be corrected easily with the NHP correction method using the POSIT algorithm.

position of the PGS region on the sagittal images were measured and analyzed (Figs. 1, 2).

Observation of the bone structures of the maxillary tuberosity and pterygomaxillary suture using cone beam CT images R. Asaumi1, T. Kawai1, I. Sato2 1 School of Life Dentistry at Tokyo, Nippon Dental University, Department of Oral & Maxillofacial Radiology, Tokyo, Japan 2 School of Life Dentstry at Tokyo, Nippon Dental University, Department of Anatomy, Tokyo, Japan

Fig. 1 Coronal image

Keywords Implant  Maxillary tuberosity  CBCT  Bone Purpose Recently, implant treatment has become a common procedure for functional rehabilitation in patients with missing teeth. However, the heavy loss of bone and the expansion of the maxillary sinus (MS) cause difficulty when placing an implant in the maxillary molar region. Therefore, bone transplantation and a sinus lift procedure are sometimes applied, but these procedures are very time-consuming. Accordingly, implants are placed in the maxillary tuberosity (MT) or use the pterygomaxillary suture (PGS) for support. Not only bone quantity but also bone quality is important to get satisfactory results. It has been reported that the thickness of the cortical bone is important to provide sufficient support for the implant. It is necessary to understand the anatomical structure before operating. Therefore, the purpose of this study was to analyze the bone structure of the MT and PGS regions using cone beam CT (CBCT) images to see if this would be of assistance in preoperative diagnosis and treatment planning. Methods Twenty Japanese cadavers in the Department of Anatomy, School of Life Dentistry at Tokyo, Nippon Dental University were used in this study. The Frankfort horizontal plane of the cadavers was set parallel to the floor. The CBCT system used for scanning images was the AZ300CT, Asahi Roentgen Industry, Ltd, Japan. The scan conditions were as follows; the tube voltage was 85 kV, the tube current was 4 mA, the scanning time was 17 s, the field of view (FOV) was 79 mmu 9 80 mm H, and the voxel size was 0.155 mm 9 0.155 mm 9 0.155 mm. Imaging software (Mimics, Materialize, Leuven, Belgium) was used to construct the CBCT images from the CBCT data. Coronal images at 5 and 10 mm anteriorly from the most posterior position of the PGS and sagittal images of the PGS were created. The cortical bone width and the grayscale value (256 shades) of the cortical and cancellous bone at 1 and 5 mm below the floor of the MS on the coronal images and the grayscale value of the PGS and cancellous bone at 1 and 5 mm above the lowest

Fig. 2 Sagittal image Results There were no significant differences in the cortical bone thickness at each measuring point. The total average cortical bone thickness was 0.8 mm. The distance from the bottom of the MS to the top of the alveolar bone on 6 sides was less than than 5 mm, because of heavy bone resorption. There was no significant difference in the grayscale values of each measuring point.. The total average grayscale value of cortical bone was 120 and that of cancellous bone was 80. The boundary between the cortical bone and cancellous bone was unclear in some cases. Regarding the PGS region, it was recognized at 1 mm above the lowest position of the PGS, but in some images the PGS was not clear at 5 mm above the lowest position of the PGS.

123

Int J CARS Conclusion It is apparent that it is difficult to obtain bone support for implants in the maxillary posterior region compared with the mandibular molar region, because the border between cortical bone and cancellous bone in some case was unclear, and the thickness of the cortical bone in the MT region was less than that in the mandibular molar region (data

123

from our previous study). It is important to bear this in mind in case of implant placement, because the internal bone structure is complicated. The results of our current study suggest that it is necessary to examine carefully not only the anatomical shape of the target bony structures, but also the internal morphology of these structures.