package np

  1. Overview
  2. Docs

Module NumpyRaw.DataSourceSource

Sourcetype tag = [
  1. | `DataSource
]
Sourcetype t = [ `DataSource | `Object ] Obj.t
Sourceval of_pyobject : Py.Object.t -> t
Sourceval to_pyobject : [> tag ] Obj.t -> Py.Object.t
Sourceval create : ?destpath:[ `S of string | `None ] -> unit -> t

DataSource(destpath='.')

A generic data source file (file, http, ftp, ...).

DataSources can be local files or remote files/URLs. The files may also be compressed or uncompressed. DataSource hides some of the low-level details of downloading the file, allowing you to simply pass in a valid file path (or URL) and obtain a file object.

Parameters ---------- destpath : str or None, optional Path to the directory where the source file gets downloaded to for use. If `destpath` is None, a temporary directory will be created. The default path is the current directory.

Notes ----- URLs require a scheme string (``http://``) to be used, without it they will fail::

>>> repos = np.DataSource() >>> repos.exists('www.google.com/index.html') False >>> repos.exists('http://www.google.com/index.html') True

Temporary directories are deleted when the DataSource is deleted.

Examples -------- ::

>>> ds = np.DataSource('/home/guido') >>> urlname = 'http://www.google.com/' >>> gfile = ds.open('http://www.google.com/') >>> ds.abspath(urlname) '/home/guido/www.google.com/index.html'

>>> ds = np.DataSource(None) # use with temporary file >>> ds.open('/home/guido/foobar.txt') <open file '/home/guido.foobar.txt', mode 'r' at 0x91d4430> >>> ds.abspath('/home/guido/foobar.txt') '/tmp/.../home/guido/foobar.txt'

Sourceval abspath : path:string -> [> tag ] Obj.t -> string

Return absolute path of file in the DataSource directory.

If `path` is an URL, then `abspath` will return either the location the file exists locally or the location it would exist when opened using the `open` method.

Parameters ---------- path : str Can be a local file or a remote URL.

Returns ------- out : str Complete path, including the `DataSource` destination directory.

Notes ----- The functionality is based on `os.path.abspath`.

Sourceval exists : path:string -> [> tag ] Obj.t -> bool

Test if path exists.

Test if `path` exists as (and in this order):

  • a local file.
  • a remote URL that has been downloaded and stored locally in the `DataSource` directory.
  • a remote URL that has not been downloaded, but is valid and accessible.

Parameters ---------- path : str Can be a local file or a remote URL.

Returns ------- out : bool True if `path` exists.

Notes ----- When `path` is an URL, `exists` will return True if it's either stored locally in the `DataSource` directory, or is a valid remote URL. `DataSource` does not discriminate between the two, the file is accessible if it exists in either location.

Sourceval open_ : ?mode:[ `R | `W | `A ] -> ?encoding:string -> ?newline:string -> path:string -> [> tag ] Obj.t -> Py.Object.t

Open and return file-like object.

If `path` is an URL, it will be downloaded, stored in the `DataSource` directory and opened from there.

Parameters ---------- path : str Local file path or URL to open. mode : 'r', 'w', 'a', optional Mode to open `path`. Mode 'r' for reading, 'w' for writing, 'a' to append. Available modes depend on the type of object specified by `path`. Default is 'r'. encoding : None, str, optional Open text file with given encoding. The default encoding will be what `io.open` uses. newline : None, str, optional Newline to use when reading text file.

Returns ------- out : file object File object.

Sourceval to_string : t -> string

Print the object to a human-readable representation.

Sourceval show : t -> string

Print the object to a human-readable representation.

Sourceval pp : Format.formatter -> t -> unit

Pretty-print the object to a formatter.