News

The SOC activities are expected to ramp up at the same time as the CAIV activities are being completed. Most of the construction staff will transfer to operations gradually during this period of early operations. Towards the end of CAIV, the Observatory is expected to reach “1^st light”. At that time, all observatory systems will have been shown to function, and IRIS, one of TMT’s first-generation instruments, will be operated together with NFIRAOS, TMT’s Adaptive Optics facility, to deliver adaptive optics corrected images. This stage marks the official end of construction of the Observatory, and most of the commissioning work still pending completion is expected to be SOC-related activities, including the commissioning of all first-generation instruments and their sub-systems, and that of all software and tools that have been developed to support the scientific operations of TMT.

 

SOC is also the phase when the first scientific data will be collected. Although the science instruments will just start their commissioning, not yet delivering their optimal performances, science data will be obtained as part of “shared-risk” science programs. The data will be submitted by the TMT scientific community to be reviewed and, if selected, implemented at the observatory. These shared-risk observations are extremely important, as they will provide the first opportunity to test, tune, and validate the operability of the full suite of observatory systems involved in the lifecycle of science programs. This includes the software tools developed for users to submit proposals, prepare and execute observations, as well as to access data from the science archive and reduce them fully, up to a stage they can be analyzed by the science teams.

 

The OCDM review also presented a complete analysis of the staffing level required to operate the TMT in its steady-state, and the associated yearly operational costs.

 

Staffing estimates were obtained both “top-down”, e.g. from comparison with other observatories and scaling for the size and complexity of TMT, and “bottom-up”, that is by defining the nature, duration and frequency of the operations tasks required to operate the facility and assess the number of personnel required. It is estimated that about 115 staff will be needed to fully operate TMT observatory 24/7 all year long.

 

The staff is distributed among the various departments of:

 

- Directorate (DIR): Covering the Observatory management, safety and communication, outreach and community engagement.

- Administration (ADM): Providing all general business support: accounting, payroll, procurement, property management, legal expertise, and human resources, which is in charge of staff recruitment, training, staff relocation, etc.

- Development Office (DEO): Management of science instruments upgrade and new instrument design.

- Systems and Projects Engineering (SPE): Covering all aspects of systems engineering and quality assurance and supporting all technical groups of the Observatory.

- Engineering operations (EOPS): Support of technical operations and maintenance of all observatory systems.

- Software and IT (SWIT): Providing all software and IT support, including network and database management, for all departments.

- Science operations (SOPS): Supporting all day and night activities related to the scientific operations of the TMT, including a group of system scientists responsible for maintaining and improving the performance of all science systems.

 

In addition to its summit location, TMT will have its main headquarters at sea-level. If built in Hawaii, which is the TMT’s preferred location, TMT summit facilities will be located at an altitude of 4050m, about 150m below the summit of Maunakea, and its sea-level headquarters will be in Hilo, about a 1.5 hour drive from the summit. TMT plans to base as many staff as possible at its sea-level location, minimizing the number of staff required to travel up the mountain daily for on-site maintenance and engineering support. In this model we expect an average of about 35 staff members will need to travel daily to the site, while the rest of the team will provide support from the main headquarters.

 

- Technical Operations:

The DEO, SPE, SWIT and EOPS departments all support technical operations, with the assistance of the safety manager and its officer, who are associated with the TMT Directorate. The bulk of the technical activities are related to the maintenance of the facility both scheduled (preventive and predictive) and reactive (or corrective), and also include engineering activities that directly support science operations (e.g., telescope and instruments checks and handover to SOPS, system calibration, etc.). While the DEO and SPE departments coordinate activities aimed at optimizing the performances of the observatory and its suite of science instruments, the EOPS and SWIT departments are fully involved in day-to-day support of its operations. Together, EOPS and SWIT represent 80% of TMT’s technical staff. EOPS is divided in groups of specific technical expertise: optics, electronics, mechanics, maintenance, AO/laser and logistics. This diverse expertise is required to maintain the highest level of reliability for the complex systems involved in the operations of the TMT, such as the control of its enclosure, telescope structure, cooling systems, and of course, the control and quality of its optics. The TMT primary mirror will be composed of 492 hexagonal segments, which will need to be regularly cleaned and recoated every two years for preserving their optimal optical performances. As a result, about 10 primary mirror segments will need to be removed and re-installed every 2 weeks to keep up with the recoating cycle. The activity of removing mirror segments, scrapping the old coating off their surface, re-depositing a fresh new coat of reflective material and re-installing segments onto the primary structure, will be a major source of technical activities all year around.

The diagram below shows how the various technical groups and SOPS nighttime crew coordinate their activity over a 24h cycle to provide complete support of TMT operations. Before the end of the observing night, a small technical crew from the SPE department will start its activities from sea-level to analyze fault reports and interact as needed with the nighttime science operations staff. The main purpose of this early shift is to update the planning of the maintenance activities based on the operational problems from the night, with the goal to have the facility fully operational for the following night. This shift is followed by the bulk of the daytime technical crew traveling up the observatory to provide preventive and corrective maintenance of the facility. This will represent the largest crew on site at any given time. This will be followed at the end of the day by a smaller technical crew that covers the first few hours of the night to secure proper handover to the SOPS staff and support the investigation and resolution of any problems that might happen at the start of night operations. Additional overnight support is provided remotely, on-demand, by technical staff located at sea-level.

- Maintenance Plan:

 

The OCDM review also provided an overview of the processes and guidelines involved in the maintenance of the facility. The goal of the TMT maintenance plan is to minimize the downtime that could affect nighttime science, while ensuring optimal performance of the observatory. This must be achieved within the limited time available during the day when maintenance activities can happen. The maintenance methodology shared during the OCDM review accounts for the maintenance activities and time required to ensure that all observatory subsystems can be operated reliably and meet their availability and maintainability over their predicted lifetime. The maintenance plan for all subsystems is then compared to the maintenance budget available to check if the required activities can be accomplished, adjusting the staffing level as necessary. TMT’s assumption is that about 35% of the maintenance time will be dedicated to reactive/corrective maintenance and the rest for preventive maintenance. Such a high level of preventive maintenance is driven by the large amount of activity involved in segment exchange/recoating.

 

The methodology used makes possible the estimation of the number of staff required for a specific maintenance budget, and also identifies the nature of the expertise required at the observatory (mechanics, optics, software, electronics, etc.) and the equipment, tools and space needed to carry out the maintenance activities. In normal operations, the SPE department is responsible for planning the maintenance activities (both daily/short-term and medium/long-term planning) and EOPS is responsible for coordinating and executing all the maintenance activities. The SPE department also provides quality assurance, responsible for monitoring the efficiency and impact of the maintenance plan on the overall operations and performance of the Observatory using a set of Key Performance Indicators.

 

The scheduling and execution of all technical activities at the site will also be supported by a crucial suite of software tools aimed at facilitating the management of the spare parts inventory, and all the equipment required to maintain the facility, as well as planning the details of all maintenance tasks, whether they are predictive, preventive, reactive or require an emergency action. These systems will be fully in place during the CAIV phase to collect important historical information about the installation, maintenance, and adjustment of all systems and parts involved in the operations of the TMT facility. In addition to spare parts and maintenance management, other critical software will be used to record fault reports, analyze engineering data, hazards and risks, and search information throughout the extensive collection of digital data collected throughout the 50+ year lifetime of the TMT.

“The software development effort at TMT will continue to add and offer new capabilities to the scientific community over the course of the observatory lifetime,” said Kim Gillies, TMT Observatory Software Architect. “TMT software teams want to improve the quality of all delivered software products and secure a reliable software system.”

TMT operations support will also rely on the DEO department for planning upgrades of existing science instruments and supporting the early design phases of its future instrumentation. This department will report directly to TMT’s director and will work closely with TMT partners to design and develop new instrumentation capabilities for TMT’s scientific community. The plan exposed at the OCDM review is to integrate a new science instrument to TMT operations roughly every three years. Considering that the timescale involved in developing a TMT instrument, from its early design to its commissioning, is of the order of 15 years in average, the DEO team will already be actively supporting the first of TMT next-generation instruments before TMT starts operating its first-light instrument suite.

 

- Science Operations:

 

Science operations encompass all phases of a proposal, from its submission and review by the Time Allocation Committee, to observation execution and finally the ingestion of science data into an archive. The current TMT model requires 23 staff to support its science operations, which includes astronomers and operations associates to support day/night operations of the telescope, science instruments and associated systems, and a group of system scientists whose focus is to support the science and technical teams in maintaining/improving the image quality and scientific performances of the facility.

 

TMT operations model will offer science observations to be executed via two main modes:

• Visitor mode: A member of the science team is present at the sea-level facility to prioritize and strategize the sequence of observations executed by the SOPS support astronomer.
• Service mode: The SOPS support astronomer carries out the observations prepared in advance by the PIs of the approved programs.

§ Eavesdropping mode: This mode is similar to the service mode, with the exception that a member of the science team is in remote communication with the SOPS support astronomer to help with real-time decisions. This mode is particularly well adapted to short programs with difficult target acquisition, or needing real-time decisions to be made.

The figure on the right represents the lifecycle of an observing proposal, from phase 1 (proposal preparation and selection), through phase 2 (observations preparation and execution) to phase 3 (data archiving and data processing).

 

For many TMT science users, the support services offered by SOPS will be the most visible display of TMT operations. With the potential addition of the U.S. government as a future partner of the US-ELT Program, NOIRLab would contribute to the development of this large set of services for TMT (and GMT) operations, which will affect all phases of science operations and will include:

 

o Phase 1: Support of the proposal submission and evaluation process.

o Phase 2: Support of the preparation and execution of the science program, including the availability of adaptive scheduling for programs executed in service mode.

§ Adaptive scheduling: In service mode, an intelligent algorithm will be developed to support the SOPS staff in selecting the next best science programs to execute, based on a set of constraints that include real-time weather conditions, science priorities, time-criticality, and completion level of science programs, as well as the need for preserving a balanced distribution of executed programs for all TMT partners.

o Phase 3: Support in automatically reducing the science data obtained and populating a science archive with the raw-, reduced- and meta- data needed for carrying out a full extent analysis of the data generated by all approved science programs.

 

 

The SOPS support staff will provide daytime and nighttime support of observations. During the day, a crew of one operation associate and two support astronomers will check the integrity of the calibrations data associated with the science observations executed the night before, support users with programs approved in service and visitor modes, work with the technical team to troubleshoot any instrument issues and prepare the instruments for the upcoming night. At night, two operation associates will be located on site to support telescope and adaptive optics operations while a support astronomer located at sea-level will execute science programs in service or visitor mode. The two staff working nighttime at the telescope will stay at the Hale Pohaku mid-level residence in daytime. This facility is located at a more comfortable altitude of 2,800 meters, where the nighttime on-site staff will reside for a maximum duration of a week.

 

In addition to the above activities, SOPS will provide a helpdesk to support science users from the TMT partnership throughout all phases of their proposal lifecycle. Although not part of TMT operations budget, it is expected that partners will establish a dedicated helpdesk to support their own community (e.g. NOIRLab is planning to have a helpdesk available to support US-ELTP users of TMT and GMT).

Considering that the main purpose of TMT will be to create scientific data of the best possible quality, the OCDM review dedicated ample time to expose the data-management plan for the TMT. The software team described the software development plan and data format to be produced, including their storage and access right, and the plans for processing the data and protecting the facility against cyberattacks.

The development of TMT core software (common software, executive software and data-management system) is managed by the Project, though software development is an in-kind contribution of India as a TMT partner. Similarly, a large part of TMT science operation software is expected to be provided by NOIRLab, as a potential US-ELTP partner.

In terms of software development, the use of software cycles is the basis for TMT’s software development and quality control processes. All software development occurs within fixed cycles called iterations. The duration of an iteration cycle is about four weeks long and has been chosen to show progress and provide adequate quality assurance feedback to the development team. An important feature of TMT software development process is its Software Test and Integration Lab (STIL), which is activated at the end of each iteration to test the quality and simulate the integration of the software code.

The data produced by the observatory are of several types and consist of science data, engineering data, metadata and ancillary files, as well as data related to the observing programs (science proposals, nighttime and software logs) and all documents required and/or generated by TMT operations. For any future users of the TMT, the most important product of TMT operations are the science data and their associated files (such as calibration files and instrument configuration files). It is estimated that the suite of first-light instruments will generate a maximum of about 200 TB of data yearly, including their associated calibration files. IRIS instrument – a near-infrared spectro-imager fed by TMT’s AO facility NFIRAOS – is expected to be the main contributor of science data produced at the Observatory. To these data, one must add the set of engineering data produced daily, which are estimated to amount to 130 TB yearly.

All science and engineering data will be archived over the expected lifetime of the observatory. The science data will be stored into the US-ELTP science archive and made accessible to users by sophisticated access and search tools, all compliant with International Virtual Observatory Alliance (IVOA) standards. The calibration files and some metadata will be immediately accessible to the public while the scientific data generated by the TMT instruments will be protected and accessible only to the program PI for 18 months, corresponding to the duration of the data proprietary period. After this, all science data will become public. It is planned that the data will be automatically transferred to the US-ELT science archive immediately after they have been generated at the Observatory. They will then be processed using the most recent calibration files associated with the instrument configuration used to collect the scientific frames.

The material presented at the OCDM review demonstrated that TMT has planned its operations with a high-level of details. TMT operations plans will continue being refined, considering the needs of all TMT partners. Nevertheless all OCDM reviewers were convinced that TMT will provide a highly efficient observatory to its worldwide community of science users.

 

---