The Visible Embryo Project (VEP) is a multi-institutional, multidisciplinary research project originally created in the early 1990s as a collaboration between the Developmental Anatomy Center at the National Museum of Health and Medicine and the Biomedical Visualization Laboratory (BVL) at the University of Illinois at Chicago, "to develop software strategies for the development of distributed biostructural databases using cutting-edge technologies for high-performance computing and communications (HPCC), and to implement these tools in the creation of a large-scale digital archive of multidimensional data on normal and abnormal human development."[1] This project related to BVL's other research in the areas of health informatics, educational multimedia, and biomedical imaging science.[2][3][4] Over the following decades, the list of VEP collaborators grew to include over a dozen universities, national laboratories, and companies around the world.
An early (1993) goal of the project was to enable what it called "Spatial Genomics," to create tools and systems for three-dimensional morphological mapping of gene expression, to correlate data from the Human Genome Project with the multidimensional location of genomic expression activity within the morphological context of organisms. This led to the invention in the late 1990s by VEP collaborators of the first system for Spatial transcriptomics.[5][6] Other areas that VEP researchers pioneered include early web technologies, cloud computing, blockchain, and virtual assistant technology.
Early history edit
The VEP was created in 1992 as a collaboration between the UIC Biomedical Visualization Laboratory, directed by Michael Doyle, and the Human Developmental Anatomy Center at the National Museum of Health and Medicine (NMHM), directed by Adrianne Noe. Doyle had been appointed to the oversight committee of the Visible Human Project at the National Library of Medicine, but it would be several years before that data would become available. Looking for other sources of high-resolution volume data on the human anatomical structure, he came across the Carnegie Collection of Human Embryology,[7] housed at the NMHM. During a sabbatical working on methods for magnetic resonance microscopy (MRM) in the laboratory of Paul Lauterbur, 2003 Nobel Laureate, Dr. Doyle created a plan for the VEP and worked with Dr. Noe to recruit a large group of prominent researchers to join as initial collaborators.[8][9]
A primary goal of the project was to provide a testbed for the development of new technologies, and the refinement of existing ones, for the application of high-speed, high-performance computing and communications to current problems in biomedical science.[10][11][12][13][14][15]
Data edit
Much of the early work involved creating serial section reconstructions from microscope slides and extracting volumetric data from the NMHM specimens, rather than just surface data. Sets of serial microscopic cross-sections through human embryos (prepared by Carnegie Collection contributors between the 1890s and 1970s) were used as sample image data around which to design and implement various components of the system. These images were digitized and processed to create 3D voxel datasets representing embryonic anatomy. Standard techniques for 3D volume visualization could then be applied to these data.[1][3] Image processing of these data was required to correct for certain artifacts that were found in the original microscope sections from routine histological techniques of the tissue preparation.
Later activities of the project would make use of MRM datasets acquired from the NMHM collection, ultra-high resolution histology images,[16] and three-dimensional adult image data acquired via the Visible Human Project, in addition to embryo data.
Collaborations edit
The VEP became a far-reaching collaborative research program involving a large number of eminent scientists across the nation and around the world, including, among many others, Michael Doyle, of UIC, then UCSF, and project founder, Adrianne Noe, Director of the National Museum of Health and Medicine, George Washington University's Robert Ledley, inventor of the Full-body CT scanner, UIUC's Paul Lauterbur, MRI pioneer and Nobel laureate, LSU's Ray Gasser, eminent embryologist, Oregon Health & Science University's Kent Thornburg, internationally renown developmental biologist, Regan Moore, Director of the DICE group at the San Diego Supercomputer Center, William Lennon of Lawrence Livermore National Laboratory, Ingrid Carlbom of Digital Equipment Corporation's Cambridge Research Lab, and Demetri Terzopoulos of the University of Toronto.
Some notable Visible Embryo Project collaborations include:
Muritech edit
In the mid-1990s, Michael Doyle collaborated with Harvard's Betsey Williams to create an internet atlas of mouse development, in a project named "Muritech." A prototype two- and three-dimensional color atlas of mouse development was developed, using two embryos, a 13.5 d normal mouse embryo and a PATCH mutant embryo of the same age. Serial sections of the embryos, with an external registration marker system, introduced into the paraffin embedding process, were prepared by standard histological methods. For the 2D atlas, color images were digitized from 100 consecutive sections of the normal embryo. For the 3D atlas, 300 gray-scale images digitized from the mutant embryo were conformally warped and reconstructed into a 3D volume dataset. The external fiducial system facilitated the three-dimensional reconstruction by providing accurate registration of consecutive images and also allowed for precise spatial calibration and the correction of warping artifacts. The atlases, with their associated anatomical knowledge base, were then integrated into a multimedia online information resource via the VEP's Web technology to provide research biologists with a set of advanced tools to analyze normal and abnormal murine embryonic development.[17]
Next-Generation Internet Contract edit
The Human Embryology Digital Library and Collaboratory Support Tools project was begun in 1999 as a demonstration of the biomedical application potential of the Next Generation Internet (NGI). The collaborators included eight organizations at sites around the continental USA, a mix of medical and information technology organizations, including George Mason University, Eolas, the Armed Forces Institute of Pathology, Johns Hopkins University, Lawrence Livermore National Laboratory, the Oregon Health & Science University, the San Diego Supercomputer Center, and the University of Illinois at Chicago. The project undertook three major applications, based on the Carnegie Collection of Embryos at the AFIP's National Museum of Health and Medicine Human Development Anatomy Center (HDAC), a collection of cellular-level tissue slides that is one of the world's largest repositories of human embryos.
These applications included:
1. Digitization, curation, and annotation of embryo data: The VEP team created a production digitization capability, using automated digital microscopy, with data automatically registered for tiling and transmitted to the repository at the San Diego Supercomputer Center, and annotated by teams of biomedical volunteers with expert-level quality control. 2. Distributed embryology education using materials derived from the Carnegie Collection to create animations of embryo development and recorded master classes that can be streamed over the Internet or downloaded to create a portable electronic classroom. 3. Clinical management planning where medical professionals and expectant parent patients can review normal and abnormal development patterns with collaborative consultation from distant experts.[9]
AnatLab edit
To enable new ways to interactively explore the VEP's massive volume datasets, Michael Doyle created the zMap system, using the Visible Human Project image data for the first prototype. In 2011, Doyle collaborated with Steven Landers, Maurice Pescitelli, and others to use zMap to create an interactive tool that allows the user to select desired sets of anatomical structures for the automated generation of 3D Quicktime VR visualizations. The system used the resources available in the Eolas AnatLab knowledgebase, which has over 2200 structures identified involving a total of over 4600 sections and 700,000 annotations overall, to access the anatomical structure surface information for individual structures. This surface information was then used to automatically extract the contained volumetric image data and convert the data into a format compatible with the Osirix volume imaging system. Automated scripts then controlled Osirix in the creation of a 3D visualization of the group of selected anatomical structures. Photorealistic results were obtained by using the original color voxel information from the original Visible Human cryosection images to color the surface of the 3D reconstruction. The system then automatically progressed through a pre-defined set of rotations to generate the set of image frames required to create a Quicktime VR (QTVR) interactive movie. This system thereby allowed an anatomy instructor to quickly and easily generate customized interactive 3D reconstructions for use in the classroom.[18]
Technologies edit
Over the decades since it was begun, the work done in the Visible Embryo Project has led to the development of several important technological breakthroughs that have had a worldwide impact:
Spatial transcriptomics edit
Even though spatial mapping of Omics data had been described as an initial goal of the VEP, it wasn't until 1999 that four VEP collaborators, Michael Doyle, George Michaels, Maurice Pescitelli, and Betsey Williams worked together to create a system for what they called "spatial genomics."[6] Today, this technology is known as Spatial transcriptomics. As their 2001 U.S. patent application states,[6] their system solved the need "to gather gene expression data in a manner that supports the type of exploratory research that can take advantage of the broad-spectrum types of biologic activity analysis enabled by today's microarray tools," as well as the need for "technology to allow the collection of large volumes of these types of data, to enable exploratory investigations into patterns of biologic activity ... to correlate gene expression data with morphological structure in a useful and easy to understand manner, such as in a volume visualization environment ... to allow the collection of larger volumes of gene expression data across a wider spectrum of gene types than ever before."
They named their system SAGA, short for Spatial Analysis of Genomic Activity. As described in the related U.S. patents,[6][19][20] the SAGA system enabled the multidimensional morphological reconstruction of tissue biologic activity and "makes it possible for biological tissue specimens to be imaged in multiple dimensions to allow morphological reconstruction. The same tissue specimen is physically sampled in a regular raster array, so that tissue samples are taken in a regular multidimensional matrix pattern across each of the dimensions of the tissue specimen. Each sample is isolated and coded so it can be later correlated to the specific multidimensional raster array coordinates, thereby providing a correlation with the sample's original pre-sampling morphological location in the tissue specimen. Each tissue sample is then analyzed with broad-spectrum biological activity methods, providing information on a multitude of biologic functional characteristics [mRNA, etc.] for that sample. The resultant raster-based biological characteristic data may then be spatially mapped into the original multidimensional morphological matrix of image data. ... various types of analysis may then be performed on the resultant correlated multidimensional spatial datasets."
Spatial transcriptomics was named the "Method of the Year for 2020" by Nature, in January 2021.[21]
The cloud edit
In 1993, Dr. Doyle became the Director of the UCSF Center for Knowledge Management (CKM). To create the underlying software and hardware that would provide the needed computational power for the VEP, Doyle's CKM group designed a new paradigm for performing remote client-server volume visualization over the Internet.[10] This involved creating a system for remotely computing visualizations through a networked cluster, or cloud, of distributed heterogeneous computational engines, and coordinating the computations to pass user interface control messages to those engines, causing the cloud computers to generate new rendered visualizations and stream the resulting views to the users' desktops, while delta-encoding and compressing the streamed data to optimize performance over low-bandwidth connections.[22]
To hide the complexity of the system from the user, they modified one of the earliest versions of the NCSA Mosaic Web browser[23] to allow their interactive cloud-computing applications to be automatically launched and run embedded within Web pages, so any user would need only to load a Web document from the VEP and would be able to immediately interactively explore the project's multidimensional datasets, rather than static representations of those datasets.[24]
In November 1993, the CKM's VEP research group demonstrated this system, the first Web-based Cloud application platform, on-stage to a meeting of approximately 300 Bay Area SIGWEB members at Xerox PARC.[25]
Today, this capability is called "the Cloud." The VEP team's work opened the door to the potential of the Web to provide rich information resources to users, regardless of where they were located and spawned a multi-trillion-dollar industry as a result.[26]
zMap edit
Dr. Doyle then began to focus more directly on the problem of how to navigate within these complex biomedical volume datasets and developed a system for mapping the semantic identity of morphological structures within the datasets and integrating those mappings with the hypermedia linking mechanism of the Web. This led to the creation of the first three-dimensional Web image map system and was used to create a variety of online interactive reference systems for biomedical education and research throughout the 90s and beyond.[27][28]
Blockchain edit
One of the challenges for large collaborative knowledge bases is how to assure the integrity of data over a long period of time. Standard cryptographic methods that depend upon trusting a central validation authority are vulnerable to a variety of factors that can lead to data corruption, including hacking, data breaches, insider fraud, and the possible disappearance of the validating authority itself. To solve this problem for the VEP data collections, Dr. Doyle created a novel type of cryptographic system, called Transient-key cryptography. This system allows the validation of data integrity without requiring users of the system to trust any central authority, and also represented the first decentralized blockchain system, enabling the later creation of the Bitcoin system. In the mid-2000s, this technology was adopted as a national standard in the ANSI ASC X9.95 Standard for trusted timestamps.[29][30]
Virtual assistants edit
Since the mid-2000s, the VEP team has made great use of digital voice and text communications systems, to facilitate communications among geographically-distributed team members. To increase the efficiency of these communications, Michael Doyle and Steve Landers collaborated to create the Skybot system, the first AI-based mobile virtual assistant system. Skybot used the power and flexibility of AI to dramatically expand the use of messaging systems. Using Skybot, one could create a variety of programmable responses to incoming calls and chat messages. The system incorporated a state machine that could be configured to automatically trigger automated responses to various communication and user-context events. This provided the user with a surprisingly broad and powerful set of capabilities for automating mobile communication operations and pioneered the mobile intelligent-assistant product category that is now ubiquitous worldwide. [31][32]
Current status edit
Plans are underway to secure the funding necessary to expand the Visible Embryo Project to create a national resource that combines large-scale knowledgebase with advanced analytical tools in an innovative online collaborative environment to support and continue to advance the art and science of Spatial Omics.[33]
See also edit
References edit
- ^ a b Doyle, Michael D.; Ang, Cheong; Raju, Rakesh; Klein, Gary; Williams, Betsey S.; DeFanti, Thomas; Goshtasby, Ardeshir; Grzesczuk, Robert; Noe, Adrianne (1993). "Processing of cross-sectional image data for reconstruction of human developmental anatomy from museum specimens". ACM SIGBIO Newsletter. 13 (1). Association for Computing Machinery (ACM): 9–15. doi:10.1145/163424.163426. ISSN 0163-5697. S2CID 1901140.
- ^ Doyle, M. (1991). "The Metamap Process: A New Approach To The Creation Of Object Oriented Image Databases For Medical Education". Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society Volume 13: 1991. IEEE. pp. 1046–1047. doi:10.1109/iembs.1991.684333. ISBN 0-7803-0216-8.
- ^ a b Carlbom, Ingrid; Hsu, William M.; Klinker, Gudrun; Szeliski, Richard; Waters, Keith; Doyle, Michael; Gettys, Jim; Harris, Kristen M.; Levergood, Thomas M.; Palmer, Ricky; Palmer, Larry; Picart, Marc; Terzopoulos, Demetri; Tonnesen, David; Vannier, Michael; Wallace, Greg (1992). "Modeling and analysis of empirical data in collaborative environments". Communications of the ACM. 35 (6). Association for Computing Machinery (ACM): 74–84. doi:10.1145/129888.129893. ISSN 0001-0782. S2CID 27152196.
- ^ Doyle, M.; Majumbar, S. (1991). "The Use Of Fractal Analysis in the Screening Of Medical/dental X- Ray And Tomographic Images For Early Signs Of Osteoporosis". Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society Volume 13: 1991. IEEE. pp. 2234–2235. doi:10.1109/iembs.1991.684978. ISBN 0-7803-0216-8.
- ^ Doyle MD, Noe A, Michaels GS (2000). Oliver WR (ed.). "The Visible Embryo Project: A Platform for Spatial Genomics". Proceedings of SPIE. 28th AIPR Workshop: 3D Visualization for Data Exploration and Decision Making. 3905: 248. Bibcode:2000SPIE.3905..248D. doi:10.1117/12.384880. S2CID 85027703.
- ^ a b c d US 7613571, Doyle, Michael D.; Pescitelli, Jr., Maurice J. & Williams, Betsey S. et al., "Method and system for the multidimensional morphological reconstruction of genome expression activity", published 2009-11-03
- ^ Cohen, Jon (September 6, 2002). "Embryo Development at the Click of a Mouse". Science. 297 (5587). American Association for the Advancement of Science (AAAS): 1629. doi:10.1126/science.297.5587.1629. ISSN 0036-8075. PMID 12215620. S2CID 32502255.
- ^ Doyle, Michael D.; Ang, Cheong S.; Martin, David C.; Noe, Adrianne (1996). "The Visible Embryo Project: Embedded program objects for knowledge access, creation and management through the World Wide Web". Computerized Medical Imaging and Graphics. 20 (6). Elsevier BV: 423–431. doi:10.1016/s0895-6111(96)00040-7. ISSN 0895-6111. PMID 9007210.
- ^ a b "Final Report Contract N01-LM-0-3508 Human Embryology Digital Library and Collaboratory Support Tools" (PDF).
- ^ a b Doyle, Michael D.; Ang, Cheong S.; Martin, David C. (March 14, 1995). Rodriguez, Arturo A.; Maitan, Jacek (eds.). Embedding interactive external program objects within open-distributed hypermedia documents. Multimedia Computing and Networking 1995. Vol. 2417. SPIE. pp. 503–511. doi:10.1117/12.206076. ISSN 0277-786X.
- ^ Carlson, Bruce M. (April 15, 2002). "Embryology in the medical curriculum". The Anatomical Record. 269 (2). Wiley: 89–98. doi:10.1002/ar.10075. ISSN 0003-276X. PMID 12001215. S2CID 28874430.
- ^ "Making visible embryos: From horizons to stages". Department of History and Philosophy of Science. Retrieved March 6, 2022.
- ^ "Majoring in Curiosity – BMCC". BMCC. October 28, 2021. Retrieved March 6, 2022.
- ^ Trelease, Robert B. (October 14, 2002). "Anatomical informatics: Millennial perspectives on a newer frontier". The Anatomical Record. 269 (5). Wiley: 224–235. doi:10.1002/ar.10177. ISSN 0003-276X. PMID 12379939. S2CID 45376272.
- ^ Abdulla, R.; Blew, G. A.; Holterman, M. J. (January 20, 2004). "Cardiovascular Embryology". Pediatric Cardiology. 25 (3). Springer Science and Business Media LLC: 191–200. doi:10.1007/s00246-003-0585-1. ISSN 0172-0643. PMID 15360112. S2CID 42516001.
- ^ Doyle, Michael D.; Harrod, Kimberly R. (2018). "Netlab: A platform for crowdsourcing virtual microscopy knowledge". The FASEB Journal. 32 (S1). Wiley. doi:10.1096/fasebj.2018.32.1_supplement.635.20. ISSN 0892-6638. S2CID 249782566.
- ^ Williams, Betsey S.; Doyle, Michael D. (1996). "An internet atlas of mouse development". Computerized Medical Imaging and Graphics. 20 (6). Elsevier BV: 433–447. doi:10.1016/s0895-6111(96)00041-9. ISSN 0895-6111. PMID 9007211.
- ^ Pescitelli, MJ; Huntley, S; Theiss, E; Lilagan, C; Doyle, MD (2011). "Automated system for on-demand generation of photorealistic structure-specific interactive anatomical reconstructions". The FASEB Journal. 25 (S1). Wiley. doi:10.1096/fasebj.25.1_supplement.495.2 (inactive January 31, 2024).
{{cite journal}}: CS1 maint: DOI inactive as of January 2024 (link) - ^ US 7894997, Doyle, Michael D.; Pescitelli, Jr., Maurice J. & Williams, Betsey S. et al., "Multidimensional morphological reconstruction of genome expression activity", published 2011-02-22
- ^ US 10011864, Doyle, Michael D.; Pescitelli, Jr., Maurice J. & Williams, Betsey S. et al., "Multidimensional microdissection and morphological reconstruction of genomic or proteomic expression activity", published 2018-07-03, assigned to Eolas Technologies Inc.
- ^ Marx, Vivien (2021). "Method of the Year: spatially resolved transcriptomics". Nature Methods. 18 (1). Springer Science and Business Media LLC: 9–14. doi:10.1038/s41592-020-01033-y. ISSN 1548-7091. PMID 33408395. S2CID 230817551.
- ^ Ang, Cheong; Martin, David; Doyle, Michael (1994). "Integrated control of distributed volume visualization through the World-Wide-Web". Proceedings Visualization '94. IEEE. pp. 13–20. doi:10.1109/VISUAL.1994.346342. ISBN 9780780325210. S2CID 2598102.
- ^ "What's New With NCSA Mosaic". Index of /. October 20, 1994. Retrieved March 6, 2022.
- ^ Doyle, Michael D.; Ang, Cheong S.; Martin, David C.; Noe, Adrianne (1996). "The Visible Embryo Project: Embedded program objects for knowledge access, creation and management through the World Wide Web". Computerized Medical Imaging and Graphics. 20 (6): 423–431. doi:10.1016/S0895-6111(96)00040-7. PMID 9007210.
- ^ Announcement: Second SIG-Web meeting (Palo Alto, CA)
- ^ US 9195507, Doyle, Michael D.; Martin, David C. & Ang, Cheong S., "Distributed hypermedia method and system for automatically invoking external application providing interaction and display of embedded objects within a hypermedia document", published 2015-11-24, assigned to Eolas Technologies Inc.
- ^ Doyle, Mike; Klein, G.; Hussaini, F.; Pescitelli, M. (May 5, 2000). Oliver, William R. (ed.). MultiVIS: a web-based interactive remote visualization environment and navigable volume imagemap system. 28th AIPR Workshop: 3D Visualization for Data Exploration and Decision Making. Vol. 3905. SPIE. pp. 173–182. doi:10.1117/12.384871. ISSN 0277-786X.
- ^ US 7792862, Doyle, Michael D., "Method and apparatus for identifying features of multidimensional image data in hypermedia systems", published 2010-09-07, assigned to Eolas Technologies Inc.
- ^ US 6381696, Doyle, Michael D., "Method and system for transient key digital time stamps", published 2002-04-30, assigned to Proofspace Inc.
- ^ US 7047415, Doyle, Michael D.; Doyle, Paul F. & Bernsohn, Glenn W. et al., "System and method for widely witnessed proof of time", published 2006-05-16, assigned to DFS Linkages Inc.
- ^ Landers, Steve; Doyle, Mike (October 20, 2008). Skybot (PDF). Proceedings of the 15th Annual Tcl/Tk Conference.
- ^ US 10015318, Landers, Steven Carl & Doyle, Michael D., "Automated communication response system", published 2018-07-03, assigned to Eolas Technologies Inc.
- ^ Branch, NIAMS Science Communications and Outreach (September 19, 2019). "Strategic Plan Fiscal Years 2020-2024". National Institute of Arthritis and Musculoskeletal and Skin Diseases. Retrieved November 7, 2023.