Loading...
Please wait, while we are loading the content...
Similar Documents
MeerKAT Control and Monitoring - Design Concepts and Status
| Content Provider | Semantic Scholar |
|---|---|
| Author | Heever, L. Van Den |
| Copyright Year | 2013 |
| Abstract | This presentation gives a status update of the MeerKAT Control & Monitoring (CAM) subsystem focusing on the design concepts and key design decisions. The presentation is supplemented by a poster of the current KAT-7 CAM system (including a demo). The vision for MeerKAT includes to: a) use Offset Gregorian antennas in a radio telescope array combined with optimized receiver technology in order to achieve superior imaging and maximum sensitivity, b) be the most sensitive instrument in the world in L-band, c) be an instrument that will be considered the benchmark for performance and reliability by the scientific community at large, and d) be a true precursor for the SKA that will be integrated into the SKA-mid dish array. MEERKAT PROJECT UPDATE KAT-7, 7-dish engineering prototype for MeerKAT, is already producing exciting science and is being operated 24x7. The first MeerKAT antenna will be on site by the end of this year and the first two Receptors will be fully integrated and ready for testing by April 2014. By December 2016 hardware for all 64 receptors will be installed and accepted and 32 antennas will be fully commissioned. With regards the MeerKAT CAM subsystem we have completed the MeerKAT CAM Preliminary Design Review in July [1] & [2] with an international panel of domain experts, and the MeerKAT CAM team is currently developing the MeerKAT CAM requirements [3] for the MeerKAT Receptor Test System (first 4 receptors) to be ready by Jan 2014. EVOLUTION OF MEERKAT CAM DESIGN SKA South Africa started in 2004 with an XDM project, which was followed by the Fringe Finder project (the first 2 KAT-7 antennas), completed by end 2009. Then the full KAT-7 project followed and all 7 antennas are currently fully operational 24x7. SKA SA are now busy with the MeerKAT project (64 antennas in the Karoo by 2016). Many people were involved over the course of these projects in providing ideas for improvements and enhancements of the CAM subsystem as part of the CAM team and other teams in the project. The current MeerKAT CAM design is a result from all these efforts, a learning process on all these project, maturing our understanding of the requirements for a remotely operated radio telescope and most recently a concerted design effort to fully document and formally review the envisaged MeerKAT CAM design with a view towards potential scalability to the size of SKA Phase 1. MEERKAT CAM DESIGN CONCEPTS AND KEY DESIGN DECISIONS This section highlights the key concepts and subsystem-wide design decisions of the MeerKAT CAM subsystem. Homogenous Node Management The CAM subsystem ensures homogeneous node management (the term “node” is used for a virtualized container running a configured set of CAM processes). • CAM nodes are virtualised across various physical servers (CAM hosts). The same suite of software is deployed on each CAM node. • A single headnode is identified as the system controller for the set of nodes and has the only copy of the active configuration that is served from the headnode to all CAM nodes. • Each CAM node (including the headnode) initially runs only a katnodemanager service that waits for the system controller to register the subset of CAM processes to run on that node (as retrieved from the active configuration on the headnode). • This allows for seamless scaling when performance demands it as it is extremely easy to add new servers that host more nodes and distribute the processes amongst the new nodes. The only action required is updating the active configuration to identify the new nodes and the processes to run at each, and then restarting the system. KATCP for Standardised Communications Underlying the CAM concept of KAT-7 and MeerKAT is a standardized communications, reporting, controlling and logging protocol, the KAT communications protocol (KATCP) [4] & [5]. The KATCP protocol is a text based, human-readable protocol build on TCP/IP and supports flexible, run-time configuration by providing interrogation of sensors and requests/commands; these are used by the clients to discover the configuration of devices dynamically. The KATCP protocol is specified as the CAM interface for all subcontracted and internal hardware devices and subsystems, as well as internal communication between CAM components. In cases where the subcontractor cannot deliver a KATCP interface, a Device Translator is Proceedings of ICALEPCS2013, San Francisco, CA, USA MOCOAAB06 Project Status Reports ISBN 978-3-95450-139-7 19 C op yr ig ht c ○ 20 14 C C -B Y3. 0 an d by th e re sp ec tiv e au th or s implemented by the CAM team to translate its specific protocol (like modbus, OPC, web-services, Ganglia metrics) to KATCP. A key concept underlying the CAM implementation is the support provided in the KATCP protocol for different sensor strategies (sampling rates) per client. This enables each component to request sensor updates at the rate required by that component, e.g. the kataware component uses a different sampling rate to generate alarms than the katmonitor components use to archive historical sensor data . Another key concept in KATCP is standardised logging. This allows devices that do not have access to storage to forward logs over their KATCP interface to the client. The log levels are standardized and the kind of information expected at each level is prescribed. The level of the KATCP logs to send over the interface can be set through the KATCP interface. In summary: • KATCP supports dynamic discovery through interrogation of monitoring points (KATCP sensors) and commands (KATCP requests) • Interrogation of KATCP sensors provide details such as a description of the monitoring point, unit of measure, nominal, warning and error ranges, and absolute min/max values on sensors. • Interrogation of KATCP requests includes help on parameters and usage examples. • KATCP defines sensor sampling to always be a value/status combination where the status can be one of a defined set of status values e.g. nominal, warning, error, failure, inactive or unreachable (each of which is well-defined). Each sensor update is also time-stamped to indicate the time at which the sensor value was refreshed by the device. • Interrogation also provides build state and version information. • The KATCP guidelines defines standardized logging levels and logging format. • KATCP is publicly released as a Python package on PyPi. • KATCP devices support multiple connections • KATCP sensors are exposed to all clients, but KATCP supports flexible reporting strategies per client for all sensors. Each client can define its own update strategy for sensors on a KATCP interface; e.g. periodic with a time period, event (on change), periodic but limited to a maximum update rate, etc. KATCP is used for all CAM component, subsystem and hardware interfacing in MeerKAT. Most Interprocess communications internal to the CAM subsystem is also implemented through KATCP interfaces. Standardised Central Logging The KATCP guidelines also specifies standardized logging for devices/subsystems. The CAM proxy layer, gathers and stores all KATCP logs centrally. The level of logging exposed on each KATCP interface is configurable via KATCP request. This provides a consistent mechanism and formatting for system-wide logs and a central store of system logs to support fault finding and engineering tests. The CAM provides a web interface for viewing on-line system logs, which allows the log sources to be filtered by source and level by the user. Proxy Layer and KATCP Device Translators All hardware devices and MeerKAT subsystems are protected from direct access through a layer of proxies implemented by the CAM subsystem. All engineering/support/system components/tools connect via the proxy layer and not directly to hardware devices/subsystems. A proxy may expose one or more devices/subsystems, and the proxy layer provides a consistent level of aggregate monitoring information for all its hardware devices/subsystems. A proxy may implement special configuration/control for a device (e.g. the Receptor proxy implements pointing corrections for antenna pointing, and the Data proxy implements delay calculations and gain corrections for the Correlator Beamformer (CBF)). The proxy layer also gathers the KATCP logs from devices and passes it on as device-logs through Python logging to a centralized logger that stores and displays all logs centrally. With the Device Translators and specification of KATCP interface for all subcontractors and subsystems, the complete instrument is managed, controlled, monitored and logged in a standard way, and exposed to the rest of the system through the proxy layer. This allow for the CAM team to develop a Simulator that simulates the KATCP interface and device behavior for each device/subsystem, so that the complete CAM system can be functionally exercised and qualified in a fully simulated environment. Fully Simulated System The CAM subsystem implements a fully simulated system up to the KATCP interface of each hardware device and subsystem. It is possible to run a configuration including only simulated devices, or any combination of real and simulated devices combined. This allows full software development, unit testing and integration testing, including CAM subsystem qualification testing without dependency on the availability of hardware. Although the full KATCP interface for each device is implemented in the simulators, the actual functionality of all the hardware components are not fully implemented; each simulator implements behaviour to the level required by CAM integration testing. However, antenna pointing and modes are simulated with realistic timing, and a MOCOAAB06 Proceedings of ICALEPCS2013, San Francisco, CA, USA ISBN 978-3-95450-139-7 20 C op yr ig ht c ○ 20 14 C C -B Y3. 0 an d by th e re sp ec tiv e au th or s Project Status Reports representative simulation of the data output of the correlator are implemented. While the CAM team is resp |
| File Format | PDF HTM / HTML |
| Alternate Webpage(s) | http://accelconf.web.cern.ch/AccelConf/ICALEPCS2013/papers/mocoaab06.pdf |
| Language | English |
| Access Restriction | Open |
| Content Type | Text |
| Resource Type | Article |
