.. include:: references.txt .. _astropy-table: ***************************** Data Tables (`astropy.table`) ***************************** Introduction ============ `astropy.table` provides functionality for storing and manipulating heterogeneous tables of data in a way that is familiar to `numpy` users. A few notable capabilities of this package are: * Initialize a table from a wide variety of input data structures and types. * Modify a table by adding or removing columns, changing column names, or adding new rows of data. * Handle tables containing missing values. * Include table and column metadata as flexible data structures. * Specify a description, units and output formatting for columns. * Interactively scroll through long tables similar to using ``more``. * Create a new table by selecting rows or columns from a table. * Perform :ref:`table_operations` like database joins, concatenation, and binning. * Maintain a table index for fast retrieval of table items or ranges. * Manipulate multidimensional columns. * Handle non-native (mixin) column types within table. * Methods for :ref:`read_write_tables` to files. * Hooks for :ref:`subclassing_table` and its component classes. Currently `astropy.table` is used when reading an ASCII table using `astropy.io.ascii`. Future releases of Astropy are expected to use the |Table| class for other subpackages such as `astropy.io.votable` and `astropy.io.fits` . Getting Started =============== The basic workflow for creating a table, accessing table elements, and modifying the table is shown below. These examples show a very simple case, while the full `astropy.table` documentation is available from the :ref:`using_astropy_table` section. First create a simple table with three columns of data named ``a``, ``b``, and ``c``. These columns have integer, float, and string values respectively:: >>> from astropy.table import Table >>> a = [1, 4, 5] >>> b = [2.0, 5.0, 8.2] >>> c = ['x', 'y', 'z'] >>> t = Table([a, b, c], names=('a', 'b', 'c'), meta={'name': 'first table'}) If you have row-oriented input data such as a list of records, use the ``rows`` keyword. In this example we also explicitly set the data types for each column:: >>> data_rows = [(1, 2.0, 'x'), ... (4, 5.0, 'y'), ... (5, 8.2, 'z')] >>> t = Table(rows=data_rows, names=('a', 'b', 'c'), meta={'name': 'first table'}, ... dtype=('i4', 'f8', 'S1')) There are a few ways to examine the table. You can get detailed information about the table values and column definitions as follows:: >>> t # doctest: +IGNORE_OUTPUT_3 a b c int32 float64 str1 ----- ------- ---- 1 2.0 x 4 5.0 y 5 8.2 z You can also assign a unit to the columns. If any column has a unit assigned, all units would be shown as follows:: >>> t['b'].unit = 's' >>> t # doctest: +IGNORE_OUTPUT_3
a b c s int32 float64 str1 ----- ------- ---- 1 2.0 x 4 5.0 y 5 8.2 z Finally, you can get summary information about the table as follows:: >>> t.info # doctest: +IGNORE_OUTPUT_3
name dtype unit ---- ------- ---- a int32 b float64 s c str1 A column with a unit works with and can be easily converted to an `~astropy.units.Quantity` object (but see :ref:`quantity_and_qtable` for a way to natively use `~astropy.units.Quantity` objects in tables):: >>> t['b'].quantity # doctest: +FLOAT_CMP >>> t['b'].to('min') # doctest: +FLOAT_CMP From within the IPython notebook, the table is displayed as a formatted HTML table (details of how it appears can be changed by altering the ``astropy.table.default_notebook_table_class`` configuration item): .. image:: table_repr_html.png Or you can get a fancier notebook interface with in-browser search and sort using `~astropy.table.Table.show_in_notebook`: .. image:: table_show_in_nb.png If you print the table (either from the notebook or in a text console session) then a formatted version appears:: >>> print(t) a b c s --- --- --- 1 2.0 x 4 5.0 y 5 8.2 z If you do not like the format of a particular column, you can change it:: >>> t['b'].info.format = '7.3f' >>> print(t) a b c s --- ------- --- 1 2.000 x 4 5.000 y 5 8.200 z For a long table you can scroll up and down through the table one page at time:: >>> t.more() # doctest: +SKIP You can also display it as an HTML-formatted table in the browser:: >>> t.show_in_browser() # doctest: +SKIP or as an interactive (searchable & sortable) javascript table:: >>> t.show_in_browser(jsviewer=True) # doctest: +SKIP Now examine some high-level information about the table:: >>> t.colnames ['a', 'b', 'c'] >>> len(t) 3 >>> t.meta {'name': 'first table'} Access the data by column or row using familiar `numpy` structured array syntax:: >>> t['a'] # Column 'a' 1 4 5 >>> t['a'][1] # Row 1 of column 'a' 4 >>> t[1] # Row object for table row index=1 # doctest: +IGNORE_OUTPUT_3 a b c s int32 float64 str1 ----- ------- ---- 4 5.000 y >>> t[1]['a'] # Column 'a' of row 1 4 You can retrieve a subset of a table by rows (using a slice) or columns (using column names), where the subset is returned as a new table:: >>> print(t[0:2]) # Table object with rows 0 and 1 a b c s --- ------- --- 1 2.000 x 4 5.000 y >>> print(t['a', 'c']) # Table with cols 'a', 'c' a c --- --- 1 x 4 y 5 z Modifying table values in place is flexible and works as one would expect:: >>> t['a'][:] = [-1, -2, -3] # Set all column values in place >>> t['a'][2] = 30 # Set row 2 of column 'a' >>> t[1] = (8, 9.0, "W") # Set all row values >>> t[1]['b'] = -9 # Set column 'b' of row 1 >>> t[0:2]['b'] = 100.0 # Set column 'b' of rows 0 and 1 >>> print(t) a b c s --- ------- --- -1 100.000 x 8 100.000 W 30 8.200 z Replace, add, remove, and rename columns with the following:: >>> t['b'] = ['a', 'new', 'dtype'] # Replace column b (different from in place) >>> t['d'] = [1, 2, 3] # Add column d >>> del t['c'] # Delete column c >>> t.rename_column('a', 'A') # Rename column a to A >>> t.colnames ['A', 'b', 'd'] Adding a new row of data to the table is as follows:: >>> t.add_row([-8, -9, 10]) >>> len(t) 4 You can create a table with support for missing values, for example by setting ``masked=True``:: >>> t = Table([a, b, c], names=('a', 'b', 'c'), masked=True, dtype=('i4', 'f8', 'S1')) >>> t['a'].mask = [True, True, False] >>> t # doctest: +IGNORE_OUTPUT_3
a b c int32 float64 str1 ----- ------- ---- -- 2.0 x -- 5.0 y 5 8.2 z You can include certain object types like `~astropy.time.Time`, `~astropy.coordinates.SkyCoord` or `~astropy.units.Quantity` in your table. These "mixin" columns behave like a hybrid of a regular `~astropy.table.Column` and the native object type (see :ref:`mixin_columns`). For example:: >>> from astropy.time import Time >>> from astropy.coordinates import SkyCoord >>> tm = Time(['2000:002', '2002:345']) >>> sc = SkyCoord([10, 20], [-45, +40], unit='deg') >>> t = Table([tm, sc], names=['time', 'skycoord']) >>> t
time skycoord deg,deg object object --------------------- ---------- 2000:002:00:00:00.000 10.0,-45.0 2002:345:00:00:00.000 20.0,40.0 The `~astropy.table.QTable` class is a variant of `~astropy.table.Table` in which `~astropy.units.Quantity` are used natively, instead of being converted to `~astropy.table.Column`. This means their units get taken into account in numerical operations, etc. In this class `~astropy.table.Column` is still used for all unit-less arrays (see :ref:`quantity_and_qtable` for details):: >>> from astropy.table import QTable >>> import astropy.units as u >>> t = QTable() >>> t['dist'] = [1, 2] * u.m >>> t['velocity'] = [3, 4] * u.m / u.s >>> t['flag'] = [True, False] >>> t dist velocity flag m m / s float64 float64 bool ------- -------- ----- 1.0 3.0 True 2.0 4.0 False >>> t.info() name dtype unit class -------- ------- ----- -------- dist float64 m Quantity velocity float64 m / s Quantity flag bool Column .. Note:: The **only** difference between `~astropy.table.QTable` and `~astropy.table.Table` is the behavior when adding a column that has a specified unit. With `~astropy.table.QTable` such a column is always converted to a `~astropy.units.Quantity` object before being added to the table. Likewise if a unit is specified for an existing unit-less `~astropy.table.Column` in a `~astropy.table.QTable`, then the column is converted to `~astropy.units.Quantity`. The converse is that if one adds a `~astropy.units.Quantity` column to an ordinary `~astropy.table.Table` then it gets converted to an ordinary `~astropy.table.Column` with the corresponding ``unit`` attribute. .. _using_astropy_table: Using ``table`` =============== The details of using `astropy.table` are provided in the following sections: Construct table --------------- .. toctree:: :maxdepth: 2 construct_table.rst Access table --------------- .. toctree:: :maxdepth: 2 access_table.rst Modify table --------------- .. toctree:: :maxdepth: 2 modify_table.rst Table operations ----------------- .. toctree:: :maxdepth: 2 operations.rst Indexing -------- .. toctree:: :maxdepth: 2 indexing.rst Masking --------------- .. toctree:: :maxdepth: 2 masking.rst I/O with tables ---------------- .. toctree:: :maxdepth: 2 io.rst pandas.rst Mixin columns ---------------- .. toctree:: :maxdepth: 2 mixin_columns.rst Implementation ---------------- .. toctree:: :maxdepth: 2 implementation_details.rst .. note that if this section gets too long, it should be moved to a separate doc page - see the top of performance.inc.rst for the instructions on how to do that .. include:: performance.inc.rst Reference/API ============= .. automodapi:: astropy.table