*********************************************************** Miscellaneous: HDF5, YAML, ASDF, pickle (`astropy.io.misc`) *********************************************************** The `astropy.io.misc` module contains miscellaneous input/output routines that do not fit elsewhere, and are often used by other Astropy sub-packages. For example, `astropy.io.misc.hdf5` contains functions to read/write :class:`~astropy.table.Table` objects from/to HDF5 files, but these should not be imported directly by users. Instead, users can access this functionality via the :class:`~astropy.table.Table` class itself (see :ref:`table_io`). Routines that are intended to be used directly by users are listed in the `astropy.io.misc` section. .. automodapi:: astropy.io.misc :headings: =- .. automodapi:: astropy.io.misc.hdf5 :headings: =- .. automodapi:: astropy.io.misc.yaml :headings: =- astropy.io.misc.asdf Package ============================ The **asdf** subpackage contains code that is used to serialize astropy types so that they can be represented and stored using the Advanced Scientific Data Format (ASDF). If both **asdf** and **astropy** are installed, no further configuration is required in order to process ASDF files that contain **astropy** types. The **asdf** package has been designed to automatically detect the presence of the tags defined by **astropy**. For convenience, users can write `~astropy.table.Table` objects to ASDF files using the :ref:`table_io`. See :ref:`asdf_io` below. Documentation on the ASDF Standard can be found `here `__. Documentation on the ASDF Python module can be found `here `__. Additional details for Astropy developers can be found in :ref:`asdf_dev`. .. _asdf_io: Using ASDF With Table I/O ------------------------- ASDF provides readers and writers for `~astropy.table.Table` using the :ref:`table_io`. This makes it easy to read and write ASDF files with `~astropy.table.Table` data. Basic Usage ^^^^^^^^^^^ Given a table, it is straightforward to write it out to an ASDF file:: from astropy.table import Table # Create a simple table t = Table(dtype=[('a', 'f4'), ('b', 'i4'), ('c', 'S2')]) # Write the table to an ASDF file t.write('table.asdf') The I/O registry automatically selects the appropriate writer function to use based on the ``.asdf`` extension of the output file. Reading a file generated in this way is equally simple using `~astropy.table.Table.read`:: t2 = Table.read('table.asdf') The I/O registry automatically selects the appropriate reader function based on the extension of the input file. In the case of both reading and writing, if the file extension is not ``.asdf`` it is possible to explicitly specify the reader/writer function to be used:: t3 = Table.read('table.zxcv', format='asdf') Advanced Usage ^^^^^^^^^^^^^^ The fundamental ASDF data structure is the tree, which is simply a nested combination of basic data structures (see `this `_ for a more detailed description). At the top level, the tree is a `dict`. The consequence of this is that a `~astropy.table.Table` object (or any object, for that matter) can be stored at any arbitrary location within an ASDF tree. The basic writer use case described above stores the given `~astropy.table.Table` at the top of the tree using a default key. The basic reader case assumes that a `~astropy.table.Table` is stored in the same place. However, it may sometimes be useful for users to specify a different top-level key to be used for storage and retrieval of a `~astropy.table.Table` from an ASDF file. For this reason, the ASDF I/O interface provides ``data_key`` as an optional keyword when writing and reading:: from astropy.table import Table t = Table(dtype=[('a', 'f4'), ('b', 'i4'), ('c', 'S2')]) # Write the table to an ASDF file using a non-default key t.write('foo.asdf', data_key='foo') A `~astropy.table.Table` stored using a custom data key can be retrieved by passing the same argument to `~astropy.table.Table.read`:: foo = Table.read('foo.asdf', data_key='foo') The ``data_key`` option only applies to `~astropy.table.Table` objects that are stored at the top of the ASDF tree. For full generality, users may pass a callback when writing or reading ASDF files to define precisely where the `~astropy.table.Table` object should be placed in the tree. The option for the write case is ``make_tree``. The function callback should accept exactly one argument, which is the `~astropy.table.Table` object, and should return a `dict` representing the tree to be stored:: def make_custom_tree(table): # Return a nested tree where the table is stored at the second level return dict(foo=dict(bar=table)) t = Table(dtype=[('a', 'f4'), ('b', 'i4'), ('c', 'S2')]) # Write the table to an ASDF file using a non-default key t.write('foobar.asdf', make_tree=make_custom_tree) Similarly, when reading an ASDF file, the user can pass a custom callback to locate the table within the ASDF tree. The option in this case is ``find_table``. The callback should accept exactly one argument, which is an `dict` representing the ASDF tree, and it should return a `~astropy.table.Table` object:: def find_table(tree): # This returns the Table that was stored by the example above return tree['foo']['bar'] foo = Table.read('foobar.asdf', find_table=find_table) .. _asdf_dev: Details ------- The **asdf** subpackage defines classes, referred to as **tags**, that implement the logic for serialization and deserialization of Astropy types. Users should never need to refer to tag implementations directly. Their presence should be entirely transparent when processing ASDF files. ASDF makes use of abstract data type definitions called **schemas**. The tag classes provided here are specific implementations of particular schemas. Some of the tags in Astropy (e.g., those related to transforms) implement schemas that are defined by the ASDF Standard. In other cases, both the tags and schemas are defined within Astropy (e.g., those related to many of the coordinate frames). Documentation of the individual schemas defined by Astropy can be found below in the :ref:`asdf_schemas` section. Not all Astropy types are currently serializable by ASDF. Attempting to write unsupported types to an ASDF file will lead to a ``RepresenterError``. In order to support new types, new tags and schemas must be created. See `Writing ASDF Extensions `_ for additional details. Schemas ------- Documentation for each of the individual ASDF schemas defined by Astropy can be found below. .. toctree:: :maxdepth: 2 asdf-schemas