NumPy Reference, Schemes and Mind Maps of Construction

Download NumPy Reference and more Schemes and Mind Maps Construction in PDF only on Docsity!

NumPy Reference

Release 1.

Written by the NumPy community

March 20, 2009

• 1 Array objects
  • 1.1 The N-dimensional array (ndarray)
  • 1.2 Scalars
  • 1.3 Data type objects (dtype)
  • 1.4 Indexing
  • 1.5 Standard array subclasses
  • 1.6 Masked arrays
  • 1.7 The Array Interface
• 2 Universal functions ( ufunc )
  • 2.1 Broadcasting
  • 2.2 Output type determination
  • 2.3 Use of internal buffers
  • 2.4 Error handling
  • 2.5 Casting Rules
  • 2.6 ufunc
  • 2.7 Available ufuncs
• 3 Routines
  • 3.1 Array creation routines
  • 3.2 Array manipulation routines
  • 3.3 Indexing routines
  • 3.4 Data type routines
  • 3.5 Input and output
  • 3.6 Fourier transforms (numpy.fft)
  • 3.7 Linear algebra (numpy.linalg)
  • 3.8 Random sampling (numpy.random)
  • 3.9 Sorting and searching
  • 3.10 Logic functions
  • 3.11 Binary operations
  • 3.12 Statistics
  • 3.13 Mathematical functions
  • 3.14 Functional programming
  • 3.15 Polynomials
  • 3.16 Financial functions
  • 3.17 Set routines
  • 3.18 Window functions
  • 3.19 Floating point error handling
  • 3.20 Masked array operations
  • 3.21 Numpy-specific help functions
  • 3.22 Miscellaneous routines
  • 3.23 Mathematical functions with automatic domain (numpy.emath)
  • 3.24 Matrix library (numpy.matlib)
  • 3.25 Optionally Scipy-accelerated routines (numpy.dual)
  • 3.26 Numarray compatibility (numpy.numarray)
  • 3.27 Old Numeric compatibility (numpy.oldnumeric)
  • 3.28 C-Types Foreign Function Interface (numpy.ctypeslib)
• 4 Packaging ( numpy.distutils )
  • 4.1 Modules in numpy.distutils
  • 4.2 Conversion of .src files
• 5 Numpy C-API
  • 5.1 Python Types and C-Structures
  • 5.2 Configuration defines
  • 5.3 Data Type API
  • 5.4 Array API
  • 5.5 UFunc API
  • 5.6 Numpy core libraries
• 6 Numpy internals
  • 6.1 Numpy C Code Explanations
  • 6.2 Internal organization of numpy arrays
  • 6.3 Multidimensional Array Indexing Order Issues
• 7 Acknowledgements
• Index

Release

Date March 20, 2009

This reference manual details functions, modules, and objects included in Numpy, describing what they are and what they do. For learning how to use NumPy, see also Numpy User Guide (in NumPy User Guide).

CONTENTS 1

CHAPTER

ONE

ARRAY OBJECTS

NumPy provides an N-dimensional array type, the ndarray, which describes a collection of “items” of the same type. The items can be indexed using for example N integers.

All ndarrays are homogenous: every item takes up the same size block of memory, and all blocks are interpreted in exactly the same way. How each item in the array is to be interpreted is specified by a separate data-type object, one of which is associated with every array. In addition to basic types (integers, floats, etc.), the data type objects can also represent data structures.

An item extracted from an array, e.g., by indexing, is represented by a Python object whose type is one of the array scalar types built in Numpy. The array scalars allow easy manipulation of also more complicated arrangements of data.

Figure 1.1: Figure Conceptual diagram showing the relationship between the three fundamental objects used to de- scribe the data in an array: 1) the ndarray itself, 2) the data-type object that describes the layout of a single fixed-size element of the array, 3) the array-scalar Python object that is returned when a single element of the array is accessed.

1.1 The N-dimensional array (ndarray)

An ndarray is a (usually fixed-size) multidimensional container of items of the same type and size. The number of dimensions and items in an array is defined by its shape, which is a tuple of N integers that specify the sizes of each dimension. The type of items in the array is specified by a separate data-type object (dtype), one of which is associated with each ndarray.

As with other container objects in Python, the contents of a ndarray can be accessed and modified by indexing or slicing the array (using for example N integers), and via the methods and attributes of the ndarray. Different

ndarrays can share the same data, so that changes made in one ndarray may be visible in another. That is, an ndarray can be a “view” to another ndarray, and the data it is referring to is taken care of by the “base” ndarray. ndarrays can also be views to memory owned by Python strings or objects implementing the buffer or array interfaces.

Example

A 2-dimensional array of size 2 x 3, composed of 4-byte integer elements:

>>> x = np.array([[1, 2, 3], [4, 5, 6]], np.int32) >>> type(x) <type ’numpy.ndarray’> >>> x.shape (2, 3) >>> x.dtype dtype(’int32’)

The array can be indexed using a Python container-like syntax:

>>> x[1,2] 6

For example slicing can produce views of the array:

>>> y = x[:,1] >>> y[0] = 9 >>> x array([[1, 9, 3], [4, 5, 6]])

1.1.1 Constructing arrays

New arrays can be constructed using the routines detailed in Array creation routines, and also by using the low-level ndarray constructor:

ndarray(shape[,dtype,buffer,offset,...])An array object represents a multidimensional, homogeneous array of fixed-size items. An associated data-type object describes the format of each element in the array (its byte-order, how many bytes it occupies in memory, whether it is an integer or a floating point number, etc.).

class ndarray () An array object represents a multidimensional, homogeneous array of fixed-size items. An associated data-type object describes the format of each element in the array (its byte-order, how many bytes it occupies in memory, whether it is an integer or a floating point number, etc.). Arrays should be constructed using array, zeros or empty (refer to the See Also section below). The parame- ters given here describe a low-level method for instantiating an array (ndarray(...)). For more information, refer to the numpy module and examine the the methods and attributes of an array.

Parameters shape : tuple of ints Shape of created array. dtype : data type, optional Any object that can be interpreted a numpy data type. buffer : object exposing buffer interface, optional

4 Chapter 1. Array objects

empty Create an array, but leave its allocated memory unchanged. dtype Create a data type.

Notes There are two modes of creating an array using new:

1.If buffer is None, then only shape, dtype, and order are used. 2.If buffer is an object exporting the buffer interface, then all keywords are interpreted.

No init method is needed because the array is fully initialized after the new method.

Examples These examples illustrate the low-level ndarray constructor. Refer to the See Also section for easier ways of constructing an ndarray. First mode, buffer is None:

>>> np.ndarray(shape=(2,2), dtype=float, order=’F’) array([[ -1.13698227e+002, 4.25087011e-303], [ 2.88528414e-306, 3.27025015e-309]])

Second mode:

>>> np.ndarray((2,), buffer=np.array([1,2,3]), ... offset=np.int_().itemsize, ... dtype=int) # offset = 1*itemsize, i.e. skip first element array([2, 3])

1.1.2 Indexing arrays

Arrays can be indexed using an extended Python slicing syntax, array[selection]. Similar syntax is also used for accessing fields in a record array.

See Also:

Array Indexing.

1.1.3 Internal memory layout of an ndarray

An instance of class ndarray consists of a contiguous one-dimensional segment of computer memory (owned by the array, or by some other object), combined with an indexing scheme that maps N integers into the location of an item in the block. The ranges in which the indices can vary is specified by the shape of the array. How many bytes each item takes and how the bytes are interpreted is defined by the data-type object associated with the array. A segment of memory is inherently 1-dimensional, and there are many different schemes of arranging the items of an N- dimensional array to a 1-dimensional block. Numpy is flexible, and ndarray objects can accommodate any strided indexing scheme. In a strided scheme, the N-dimensional index (n 0 , n 1 , ..., nN − 1 ) corresponds to the offset (in bytes)

noffset =

N∑ − 1

k=

sknk

6 Chapter 1. Array objects

from the beginning of the memory block associated with the array. Here, sk are integers which specify the strides of the array. The column-major order (used for example in the Fortran language and in Matlab) and row-major order (used in C) are special cases of the strided scheme, and correspond to the strides:

scolumn k =

k∏− 1

j=

dj , srow k =

N∏ − 1

j=k+

dj.

Both the C and Fortran orders are contiguous, i.e. single-segment, memory layouts, in which every part of the memory block can be accessed by some combination of the indices.

Data in new ndarrays is in the row-major (C) order, unless otherwise specified, but for example basic array slicing often produces views in a different scheme.

Note: Several algorithms in NumPy work on arbitrarily strided arrays. However, some algorithms require single- segment arrays. When an irregularly strided array is passed in to such algorithms, a copy is automatically made.

1.1.4 Array attributes

Array attributes reflect information that is intrinsic to the array itself. Generally, accessing an array through its at- tributes allows you to get and sometimes set intrinsic properties of the array without creating a new array. The exposed attributes are the core parts of an array and only some of them can be reset meaningfully without creating a new array. Information on each attribute is given below.

Memory layout

The following attributes contain information about the memory layout of the array:

ndarray.flags Information about the memory layout of the array.

ndarray.shape Tuple of array dimensions.

ndarray.strides Tuple of bytes to step in each dimension.

ndarray.ndim Number of array dimensions.

ndarray.data Buffer object pointing to the start of the data.

ndarray.size Number of elements in the array.

ndarray.itemsize Length of one element in bytes.

ndarray.nbytes Number of bytes in the array.

ndarray.base Base object if memory is from some other object.

flags Information about the memory layout of the array.

Attributes C_CONTIGUOUS (C) : The data is in a single, C-style contiguous segment. F_CONTIGUOUS (F) : The data is in a single, Fortran-style contiguous segment.

1.1. The N-dimensional array (ndarray) 7

Examples

>>> x = np.reshape(np.arange(567*8), (5,6,7,8)).transpose(2,3,1,0) >>> x.strides (32, 4, 224, 1344) >>> i = np.array([3,5,2,2]) >>> offset = sum(i * x.strides) >>> x[3,5,2,2] 813 >>> offset / x.itemsize 813

ndim Number of array dimensions.

Examples

>>> x = np.array([1,2,3]) >>> x.ndim 1 >>> y = np.zeros((2,3,4)) >>> y.ndim 3

data Buffer object pointing to the start of the data.

size Number of elements in the array.

Examples

>>> x = np.zeros((3,5,2), dtype=np.complex128) >>> x.size 30

itemsize Length of one element in bytes.

Examples

>>> x = np.array([1,2,3], dtype=np.float64) >>> x.itemsize 8 >>> x = np.array([1,2,3], dtype=np.complex128) >>> x.itemsize 16

nbytes Number of bytes in the array.

Examples

>>> x = np.zeros((3,5,2), dtype=np.complex128) >>> x.nbytes 480 >>> np.prod(x.shape) * x.itemsize 480

1.1. The N-dimensional array (ndarray) 9

base Base object if memory is from some other object.

Examples Base of an array owning its memory is None:

>>> x = np.array([1,2,3,4]) >>> x.base is None True

Slicing creates a view, and the memory is shared with x:

>>> y = x[2:] >>> y.base is x True

Note: XXX: update and check these docstrings.

Data type

See Also:

Data type objects

The data type object associated with the array can be found in the dtype attribute:

ndarray.dtype Data-type for the array.

dtype Data-type for the array.

Note: XXX: update the dtype attribute docstring: setting etc.

Other attributes

ndarray.T Same as self.transpose() except self is returned for self.ndim < 2.

ndarray.real The real part of the array.

ndarray.imag The imaginary part of the array.

ndarray.flat A 1-d flat iterator.

ndarray.ctypes A ctypes interface object.

array_priority

T

Same as self.transpose() except self is returned for self.ndim < 2.

Examples

>>> x = np.array([[1.,2.],[3.,4.]]) >>> x.T array([[ 1., 3.], [ 2., 4.]])

10 Chapter 1. Array objects

1.1.5 Array methods

An ndarray object has many methods which operate on or with the array in some fashion, typically returning an array result. These methods are explained below.

For the following methods there are also corresponding functions in numpy: all, any, argmax, argmin, argsort, choose, clip, compress, copy, cumprod, cumsum, diagonal, imag, max, mean, min, nonzero, prod, ptp, put, ravel, real, repeat, reshape, round, searchsorted, sort, squeeze, std, sum, swapaxes, take, trace, transpose, var.

Array conversion

ndarray.item() Copy the first element of array to a standard Python scalar and return it. The array must be of size one.

ndarray.tolist() Return the array as a possibly nested list.

ndarray.itemset()

ndarray.tostring([order])Construct a Python string containing the raw data bytes in the array.

ndarray.tofile(fid[,sep,format])Write array to a file as text or binary.

ndarray.dump(file) Dump a pickle of the array to the specified file. The array can be read back with pickle.load or numpy.load.

ndarray.dumps() Returns the pickle of the array as a string. pickle.loads or numpy.loads will convert the string back to an array.

ndarray.astype(t) Copy of the array, cast to a specified type.

ndarray.byteswap(inplace)Swap the bytes of the array elements

ndarray.copy([order])Return a copy of the array.

ndarray.view([dtype,type])New view of array with the same data.

ndarray.getfield(dtype,offset)Returns a field of the given array as a certain type. A field is a view of the array data with each itemsize determined by the given type and the offset into the current array.

ndarray.setflags([write,align,uic])

ndarray.fill(value)Fill the array with a scalar value.

item () Copy the first element of array to a standard Python scalar and return it. The array must be of size one.

tolist () Return the array as a possibly nested list. Return a copy of the array data as a hierarchical Python list. Data items are converted to the nearest compatible Python type.

Parameters none :

12 Chapter 1. Array objects

Returns y : list The possibly nested list of array elements.

Notes The array may be recreated, a = np.array(a.tolist()).

Examples

>>> a = np.array([1, 2]) >>> a.tolist() [1, 2] >>> a = np.array([[1, 2], [3, 4]]) >>> list(a) [array([1, 2]), array([3, 4])] >>> a.tolist() [[1, 2], [3, 4]]

itemset ()

tostring (order=’C’) Construct a Python string containing the raw data bytes in the array.

Parameters order : {‘C’, ‘F’, None} Order of the data for multidimensional arrays: C, Fortran, or the same as for the original array.

tofile (fid, sep="", format="%s") Write array to a file as text or binary. Data is always written in ‘C’ order, independently of the order of a. The data produced by this method can be recovered by using the function fromfile(). This is a convenience function for quick storage of array data. Information on endianess and precision is lost, so this method is not a good choice for files intended to archive data or transport data between machines with different endianess. Some of these problems can be overcome by outputting the data as text files at the expense of speed and file size.

Parameters fid : file or string An open file object or a string containing a filename. sep : string Separator between array items for text output. If “” (empty), a binary file is written, equivalently to file.write(a.tostring()). format : string Format string for text file output. Each entry in the array is formatted to text by convert- ing it to the closest Python type, and using “format” % item.

dump (file) Dump a pickle of the array to the specified file. The array can be read back with pickle.load or numpy.load.

Parameters file : str A string naming the dump file.

1.1. The N-dimensional array (ndarray) 13

Examples

>>> x = np.array([[1,2,3],[4,5,6]], order=’F’)

>>> y = x.copy()

>>> x.fill(0)

>>> x array([[0, 0, 0], [0, 0, 0]])

>>> y array([[1, 2, 3], [4, 5, 6]])

>>> y.flags[’C_CONTIGUOUS’] True

view (dtype=None, type=None) New view of array with the same data.

Parameters dtype : data-type Data-type descriptor of the returned view, e.g. float32 or int16. type : python type Type of the returned view, e.g. ndarray or matrix.

Examples

>>> x = np.array([(1, 2)], dtype=[(’a’, np.int8), (’b’, np.int8)])

Viewing array data using a different type and dtype:

>>> y = x.view(dtype=np.int16, type=np.matrix) >>> print y.dtype int

>>> print type(y) <class ’numpy.core.defmatrix.matrix’>

Using a view to convert an array to a record array:

>>> z = x.view(np.recarray) >>> z.a array([1], dtype=int8)

Views share data:

>>> x[0] = (9, 10) >>> z[0] (9, 10)

1.1. The N-dimensional array (ndarray) 15

getfield (dtype, offset) Returns a field of the given array as a certain type. A field is a view of the array data with each itemsize determined by the given type and the offset into the current array.

setflags (write=None, align=None, uic=None)

fill (value) Fill the array with a scalar value.

Parameters a : ndarray Input array value : scalar All elements of a will be assigned this value.

Examples

>>> a = np.array([1, 2]) >>> a.fill(0) >>> a array([0, 0]) >>> a = np.empty(2) >>> a.fill(1) >>> a array([ 1., 1.])

Note: XXX: update and check these docstrings.

Shape manipulation

For reshape, resize, and transpose, the single tuple argument may be replaced with n integers which will be interpreted as an n-tuple.

ndarray.reshape(shape[,order])Returns an array containing the same data with a new shape.

ndarray.resize(new_shape[,refcheck,order])Change shape and size of array in-place.

ndarray.transpose(*axes)Returns a view of ‘a’ with axes transposed. If no axes are given, or None is passed, switches the order of the axes. For a 2-d array, this is the usual matrix transpose. If axes are given, they describe how the axes are permuted.

ndarray.swapaxes(axis1,axis2)Return a view of the array with axis1 and axis2 interchanged.

ndarray.flatten([order])Collapse an array into one dimension.

ndarray.ravel([order]) Return a flattened array.

ndarray.squeeze() Remove single-dimensional entries from the shape of a.

reshape (shape, order=’C’) Returns an array containing the same data with a new shape. Refer to numpy.reshape for full documentation. See Also:

16 Chapter 1. Array objects