1 / 27

Automating electronic data sheets CCSDS Spring 2013

Automating electronic data sheets CCSDS Spring 2013. Sam Price. Navigator Background. Missions GPM MMS – High Altitude 12Re Hubble servicing mission Reflected GPS signals Embedded orbit propagator Geons 3 rd Party software 3 rd party commands/telemetry 45 commands

moriah
Download Presentation

Automating electronic data sheets CCSDS Spring 2013

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Automating electronic data sheetsCCSDS Spring 2013 Sam Price

  2. Navigator Background • Missions • GPM • MMS – High Altitude 12Re • Hubble servicing mission • Reflected GPS signals • Embedded orbit propagator • Geons • 3rd Party software • 3rd party commands/telemetry • 45 commands • 43 telemetry messages

  3. Navigator Electronic Documents for Command / Telemetry

  4. Multiple documents are expensive $$$$ • 15 different electronic versions, • manually created • Same data across all of the documents • 5+ different people maintaining separate documents. • Changes propagate slowly throughout documents ($$$) • Errors occur from manual changes.($$$$) • Naming conventions are not the same

  5. A streamlined approach 3rd party C Headers Some tools • C/C++ • Endianess conversion routines • Command validation • Header files • Human readable Interface Control Document • Latex • HTML • Maybe MS Word … • Ground system definitions • Bandwidth spreadsheet • Ada mockup of telemetry for testing. • Logging tools • CSV Logging • Matlab import • Database logging C++ headers Telemetry/ Commands XML Database Why C/C++ as starting point • Widespread existence • All of my Telemetry is in C • More people know C • Machine readable • Exactly what is implemented • 3rd party libraries are in C • Many more reasons

  6. Workflow Convert C/C++ to XML 3rd party C Headers Latex ICD xml_to_latex.py Asist C++ headers Telemetry/ Commands XML xml_to_asist.py Asist RDLS XML Database Python xml_to_ctypes.py Python CTypes

  7. What does it take to mimic C/C++ in XML • No child / parents / inheritance • No templates • C/C++ can have alignment/padding problems • Metadata stored in doxygen style comments • Units, Polynomials, warning ranges, etc • Bit fields defined in comments • No C bitfield yet… • Basic Types • bools • integers -- Use stdint.h • int8_t,int16_t,int32_t, int64_t • unsigned Integers • uint8_t,uint16_t,uint32_t, uint64_t • floating point memory • float, double, long double • strings • char arrays • Arrays • Multidimensional • void pointer, pointers (16,32,64) sizes • Structures/classes • unions • enums

  8. Use a compiler for parsing • Clang/llvm • C/C++ Compiler similar to GCC • Cross platform compatible(Windows,Linux,Mac) • LibClang • Pass arguments to change C/C++ structure padding. • -m32 , -m64 • Cross platform compatible (Linux, Mac, Windows) • Apple uses it in their developer tools for code completion, syntax highlighting, etc… • Use it to find comments, structures, types, padding in structure, etc. • http://clang.llvm.org/docs/Tooling.html • http://clang.llvm.org/get_started.html • https://github.com/trolldbois/clang <- padding/size info

  9. C Example CCSDS Header /*! * @brief CCSDS Telemetry Header * @TelemetryPacketHeader * */ typedefstructCCSDS_Telemetry_Header { /*! * @brief Destination address */ charp_sp_dest; /*! * @brief Protocol ID * @default 0xf2 */ charp_sp_prot; /*! *@bitarraymsbf * @bits 3 * @name packet_version_number * @brief Pacet Version Number * @default 0 * @bits 1 * @name packet_type * @brief Packet type * @enum ENUM_PACKET_TYPE * @bits 1 * @name secondary_header_present * @brief Secondary Header flag * @detailed The Secondary Header Flag shall indicate the presence or absence of the Packet Secondary Header within this Space Packet. * It shall be 1 if a Packet Secondary Header is present; it shall be 0 if a Packet Secondary Header is not present. * @enumCCSDS_Secondary_Header * @bits 11 * @name apid * @brief Application Process Identifier (APID) * @detailed via 135.0-b-1 application ID's 2040- 2047 are reserved and should not be used. * The APID (possibly in conjunction with the optional APID Qualifier that * identifies the naming domain for the APID) shall provide the naming mechanism for the LDP. */ uint16_t p_ident; /*! . . . . */ uint16_t p_seq_cont; /*! . . . . */ uint16_t p_data_len; /*! . . . . */ uint64_t s_time; } CCSDS_Telemetry_Header; Doxygen comments start with @marker_name marker value Indented comments belong to parent comment Could have used C bitfields (See backup slides)

  10. C/C++ -> clang_to_json.py -> json_to_xml.py-> XML • Current status • 500 lines of code to go from C Header to XML • Need to add enums, meta data, bit fields. • Implemented structures, unions, bit structures, arrays, padding bytes, integers, unsigned integers, floating point numbers, pointers, comment parsing. • https://github.com/nasa/SOIS-CCSDS-XML • NonNasa Contact for access, currently private project until XML format is well defined. • NASA: fill out a NAMS/IDMAX request

  11. C Example p_ident /*! *@bitarraymsbf * @bits 3 * @name packet_version_number * @brief Pacet Version Number * @default 0 * @bits 1 * @name packet_type * @brief Packet type * @bits 1 * @name secondary_header_present * @brief Secondary Header flag * @detailed The Secondary Header Flag shall indicate the presence or absence of the Packet Secondary Header within this Space Packet. * It shall be 1 if a Packet Secondary Header is present; it shall be 0 if a Packet Secondary Header is not present. * @bits 11 * @name apid * @brief Application Process Identifier (APID) * @detailed via 135.0-b-1 application ID's 2040- 2047 are reserved and should not be used. * The APID (possibly in conjunction with the optional APID Qualifier that * identifies the naming domain for the APID) shall provide the naming mechanism for the LDP. */ uint16_t p_ident;

  12. Clang to JSON Output for p_ident ….. {'array': [], 'basetype': 'uint16_t', 'basetype_name': None, 'bitarray': True, 'data': <clang.cindex.Cursor object at 0x10a7969e0>, 'is_pointer': False, 'kind': CursorKind.FIELD_DECL, 'name': 'p_ident', 'offset': 16, 'size': 2L, 'comments': [{'bitarray': 'msbf', 'children': [{'bits': '3', 'children': ['name': 'packet_version_number'}, {'brief': 'Pacet Version Number’}, {'default': '0'} ] }, ], …. More Member Comments ….. ] 'members': [{'array': [], 'basetype_name': None, 'bitarray': False, 'bits': 3, 'comments': [{'name': 'packet_version_number'}, {'brief': 'Pacet Version Number’}, {'default': '0'}], 'data': None, 'is_pointer': False, 'kind': None, 'members': [], 'name': 'packet_version_number', 'size': -1}, … More Members..], }, > python clang_to_json.py../examples/src_cpp/ccsds.h -m32 –I../examples/src_cpp • Offset, Sizes include padding information. • Clang object used for further processing if needed. • Bit arrays become children for bitstructs.

  13. XML Layout <TypeDatabase> <typeInt/> <typeFloat/> <typeBitStruct> <Members> <variable/> </Members> <typeBitStruct\> <typeStruct> <Members> <variable/> </Members> <typeStruct/> </TypeDatabase> <Telemetry> </Telemetry> <Commands> </Commands> • Type database • Basic types • Integers, floats • Structures • Unions • Bit structures • Telemetry database • Similar to structures • Command database • Similar to structures

  14. C Example /*! * @brief CCSDS Telemetry Header * @TelemetryPacketHeader * */ typedefstructCCSDS_Telemetry_Header { /*! * @brief Destination address */ charp_sp_dest; /*! * @brief Protocol ID * @default 0xf2 */ charp_sp_prot; /*! *@bitarraymsbf * @bits 3 * @name packet_version_number * @brief Pacet Version Number * @default 0 * @bits 1 * @name packet_type * @brief Packet type * @enum ENUM_PACKET_TYPE * @bits 1 * @name secondary_header_present * @brief Secondary Header flag * @detailed The Secondary Header Flag shall indicate the presence or absence of the Packet Secondary Header within this Space Packet. * It shall be 1 if a Packet Secondary Header is present; it shall be 0 if a Packet Secondary Header is not present. * @enumCCSDS_Secondary_Header * @bits 11 * @name apid * @brief Application Process Identifier (APID) * @detailed via 135.0-b-1 application ID's 2040- 2047 are reserved and should not be used. * The APID (possibly in conjunction with the optional APID Qualifier that * identifies the naming domain for the APID) shall provide the naming mechanism for the LDP. */ uint16_t p_ident; /*! . . . . */ uint16_t p_seq_cont; /*! . . . . */ uint16_t p_data_len; /*! . . . . */ uint64_t s_time; } CCSDS_Telemetry_Header;

  15. Generated XML <TypeDatabase> <typeIntbits="3"endianess="bigEndian"name="int3_be"signed="true"/> …. More small types for int1, int11, int8, int16 .. etc …. <typeBitStructname="CCSDS_Telemetry_Header_p_ident"type="uint16_be"> <Members> <variablename="packet_version_number"type="int3_be"/> <variablename="packet_type"type="int1_be"/> <variablename="secondary_header_present"type="int1_be"/> <variablename="apid"type="int11_be"/> </Members> </typeBitStruct> …. More typeBitStruct for p_seq_cont, s_time …. <typeStructname="CCSDS_Telemetry_Header"> <Members> <variablebits="8"name="p_sp_dest"offset="0"type="int8_be"/> <variablebits="8"name="p_sp_prot"offset="8"type="int8_be"/> <variablebits="16"name="p_ident"offset="16"type="CCSDS_Telemetry_Header_p_ident"/> <variablebits="16"name="p_seq_cont"offset="32"type="CCSDS_Telemetry_Header_p_seq_cont"/> <variablebits="16"name="p_data_len"offset="48"type="uint16_be"/> <variablebits="64"name="s_time"offset="64"type="CCSDS_Telemetry_Header_s_time"/> </Members> </typeStruct> </TypeDatabase> > python json_to_xml.py../examples/src_cpp/messages.h -m32 –I../examples/src_cpp

  16. Writing XML with python fromxml.etree.ElementTreeimportElement fromxml.etree.ElementTreeimportElementTree parent_node= Element('EDS') Messages = Element('Telemetry') parent_node.append(Messages) #All attributes/text must text Messages.attrib['Count'] = str(5) Messages.text = “Hello” document = ElementTree(parent_node) #See json_to_xml for prettify function print prettify(document.getroot())

  17. XML Output <?xml version="1.0" ?> <EDS> <Telemetry Count="5"> Hello </Telemetry> </EDS> Or Root = document.getroot() print xml.etree.ElementTree.tostring(Root,'utf-8') '<EDS><Telemetry Count="5">Hello</Telemetry></EDS>'

  18. XML to Python CTypes • Decode telemetry/send commands json_to_xml.py clang_to_json.py 3rd party C Headers xml_to_ctypes.py Python XML C++ headers Telemetry/ Commands Python CTypes

  19. Reading XML with python importxml.etree.ElementTreeas ET doc = ET.parse(filename) root = doc.getroot() for node inroot.findall('TypeDatabase/typeStruct'): print node.attrib['name’] print node.text See xml_to_ctypes.py for complete listing 200 lines of code to convert from XML to python ctypes Or http://docs.python.org/2/library/xml.etree.elementtree.html

  20. Python ctypes example importctypes classCCSDS_Telemetry_Header_p_ident(ctypes.LittleEndianStructure): """""" #_packed_ _fields_=[("apid",ctypes.c_uint16,11), ("secondary_header_present”,ctypes.c_uint16,1), ("packet_type",ctypes.c_uint16,1), ("packet_version_number",ctypes.c_uint16,3)] classCCSDS_Telemetry_Header(ctypes.LittleEndianStructure): ""”This is a comment about the CCSDS Packet Header""" _pack_=1 _fields_=[("p_sp_dest",ctypes.c_int8 ), ("p_sp_prot",ctypes.c_int8 ), ("p_ident",CCSDS_Telemetry_Header_p_ident), ("p_seq_cont",CCSDS_Telemetry_Header_p_seq_cont), ("p_data_len”,ctypes.c_int16), ("s_time",CCSDS_Telemetry_Header_s_time)] • python xml_to_ctypes.pytest.xml > test_ctypes.py • 205 lines of code

  21. Using python ctypes to decode / encode data importtest_ctypesastc fid=open('log.raw’) #Buffer big enough to hold any command / telemetry data big_buffer=bytearray(1024*10) write_buffer=memoryview(big_buffer) #header data directly into the buffer. 8bytes. fid.readinto(write_buffer[0:8]) #big_buffer[0:8] #bytearray(b'\x00\xf2E\n\x00\xc0\x17\x00\') header=tc.CCSDS_Telemetry_Header.from_buffer(big_buffer) #Read in the rest of the CCSDS packet. fid.readinto(write_buffer[8:(header.p_data_len+1)]) print"Apid is %d decode it."%header.p_ident.apid #changing ctypes changes the binary data. Good for commanding header.p_sp_dest=5 #big_buffer[0:8] #bytearray(b'\x05\xf2E\n\x00\xc0\x17\x00') fid.close()

  22. ctypescont… classCCSDS_Telemetry_Header(ctypes.LittleEndianStructure): "””Documentation about CCSDS Telemetry Header""" _pack_=1 _fields_=[("p_sp_dest",ctypes.c_int8), ("p_sp_prot",ctypes.c_int8), ("p_ident",CCSDS_Telemetry_Header_p_ident), ("p_seq_cont",CCSDS_Telemetry_Header_p_seq_cont), ("_p_data_len",ctypes.c_int16), ("s_time",CCSDS_Telemetry_Header_s_time)] @property defp_data_len(self): “””More Documentation””” returnself._p_data_len+1 @p_data_len.setter defp_data_len(self,new_len): self._p_data_len=new_len–1 h = CCSDS_Telemetry_Header h.p_data_len = 8 print h._p_data_len 7 print h.p_data_len 8 CCSDS data length is 1 minus actual length of the message. Provide documentation via docstring on telemetry value. Telemetry: Polynomials, conversions, etc Commanding: limit values, throw errors if not within ranges.

  23. Lessoned learned • Provide all of the standard C/C++ types • As listed on “mimic C/C++ in XML” slide • Standardized naming of Integers, floats, bools, pointers. (int32,int16,int8, …) • Allows for creating dictionary mappings between XML and desired language • Mimic the structure of existing languages. • Makes mapping from XML to existing language more strait forward • Keep it simple • Develop tools with XML standard • Kept having to change scheme while making tools to make processing easier

  24. Backup slides

  25. C bit field example(Not supporting, yet …) structCCSDS_Telemetry_header_p_ident { uint16_t apid:11; uint16_t secondary_header_present:1; uint16_t packet_type:1; uint16_t packet_version_number:1; }; http://msdn.microsoft.com/en-us/library/ewwyfdbe(v=vs.71).aspx Least significant field first

  26. C Example cont /*! * @ingroup MESSAGES * @brief An example message for Power point. * @apid 135 */ typedefstructBasic_M{ CCSDS_Telemetry_Headerpacket_header; //CCSDS Packet Header /*! * @brief Temp of the baseplate the cpu is attached to. * @polynomial [1566.7676,-18.85822,-.17372, 0.00229, 0.00001] * @unitfahrenheit * @modenominal * @red_limit [-Inf,-20), (120,Inf] * @yellow_limit [-20,-15), (115,120] */ int32_t cpu_temp; /*! * @brief Receiver Position * @coordinate_frame ECEF * @unitmeter */ doubleposition[3]; }; Doxygen comments start with @marker_name marker value Indented comments belong to parent comment

  27. Example basic example <Telemetry> <Messagename="Basic_M"> <Members> <variablebits="128"name="packet_header"offset="0"type="CCSDS_Telemetry_Header"/> <variablebits="32"name="cpu_temp"offset="128"type="int32_be"/> <variablebits="192"name="position"offset="192"type="float64_be"> <array> <size> 3 </size> </array> </variable> </Members> </Message> </Telemetry>

More Related