Apyfal API

This section describes the apyfal Python package API.

Apyfal

Copyright 2018 Accelize

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

class apyfal.Accelerator(accelerator=None, config=None, accelize_client_id=None, accelize_secret_id=None, host_type=None, host_ip=None, stop_mode=None, prefer_self_hosted=None, **host_kwargs)[source]

This class provides the full accelerator features by handling Accelerator and its host.

Parameters:
  • accelerator (str) – Name of the accelerator to initialize, to know the accelerator list please visit “https://accelstore.accelize.com”.
  • config (apyfal.configuration.Configuration, path-like object or file-like object) – If not set, will search it in current working directory, in current user “home” folder. If none found, will use default configuration values. Path-like object can be path, URL or cloud object URL.
  • accelize_client_id (str) – Accelize Client ID. Client ID is part of the access key generated on “https:/accelstore.accelize.com/user/applications”.
  • accelize_secret_id (str) – Accelize Secret ID. Secret ID come with xlz_client_id.
  • host_type (str) – Type of host to use.
  • host_ip (str) – IP or URL address of an already existing host to use. If not specified, create a new host.
  • stop_mode (str or int) – Host stop mode. Default to ‘term’ if new host, or ‘keep’ if already existing host. See “apyfal.host.Host.stop_mode” property for more information and possible values.
  • prefer_self_hosted (bool) – Default to True. If True, and current machine is an accelerator host, operates this accelerator instead of instantiating a new accelerator host. If current machine is not an accelerator host, instantiates a new host normally.
  • host_kwargs – Keyword arguments related to specific host. See targeted host class to see full list of arguments.
client

Accelerator client.

Returns:Accelerator client
Return type:apyfal.client.AcceleratorClient
host

Accelerator host.

Returns:Host
Return type:apyfal.host.Host subclass
process(src=None, dst=None, info_dict=None, **parameters)[source]

Processes with accelerator.

Parameters:
  • src (path-like object or file-like object) – Source data to process. Path-like object can be path, URL or cloud object URL.
  • dst (path-like object or file-like object) – Processed data destination. Path-like object can be path, URL or cloud object URL.
  • parameters (path-like object, str or dict) – Accelerator process specific parameters Can also be a full process parameters dictionary (Or JSON equivalent as str literal) Parameters dictionary override default configuration values, individuals specific parameters overrides parameters dictionary values. Take a look to accelerator documentation for more information on possible parameters. Path-like object can be path, URL or cloud object URL.
  • info_dict (dict or None) – If a dict passed, this dict is updated with extra information from current operation.
Returns:

Result from process operation, depending used accelerator.

process_map(srcs=None, dsts=None, timeout=None, info_list=None, **parameters)

Map process execution on multiples files.

Parameters:
  • srcs (iterable of path-like object or file-like object) – Iterable of input data to process. Must be an iterable of “src” parameters of the “process” method. Path-like object can be path, URL or cloud object URL.
  • dsts (iterable of path-like object or file-like object) – Iterable of output data. Must be an iterable of “dst” parameters of the “process” method. Path-like object can be path, URL or cloud object URL.
  • timeout (float) – The maximum number of seconds to wait. If None, then there is no limit on the wait time.
  • parameters (path-like object, str or dict) – Accelerator process specific parameters Can also be a full process parameters dictionary (Or JSON equivalent as str literal) Parameters dictionary override default configuration values, individuals specific parameters overrides parameters dictionary values. Take a look to accelerator documentation for more information on possible parameters. Path-like object can be path, URL or cloud object URL.
  • info_list (list) – If a list passed, this list is updated with “info_dict” extra information dicts for each process operation.
Returns:

Results.

Return type:

generator

Raises:

concurrent.futures.TimeoutError – “timeout” reached on at least one task.

process_running_count

Return number of asynchronous process tasks running or pending.

Returns:count.
Return type:int
process_submit(src=None, dst=None, info_dict=None, **parameters)[source]

Schedules the process operation to be executed and returns a Future object representing the execution.

See “apyfal.Accelerator.process”

Parameters:
  • src (path-like object or file-like object) – Source data to process. Path-like object can be path, URL or cloud object URL.
  • dst (path-like object or file-like object) – Processed data destination. Path-like object can be path, URL or cloud object URL.
  • parameters (path-like object, str or dict) – Accelerator process specific parameters Can also be a full process parameters dictionary (Or JSON equivalent as str literal) Parameters dictionary override default configuration values, individuals specific parameters overrides parameters dictionary values. Take a look to accelerator documentation for more information on possible parameters. Path-like object can be path, URL or cloud object URL.
  • info_dict (dict or None) – If a dict passed, this dict is updated with extra information from current operation. The dict will be updated on task completion.
Returns:

Future object representing execution.

See “apyfal.Accelerator.process” method for “Future.result()” content.

Return type:

concurrent.futures.Future

start(stop_mode=None, src=None, info_dict=None, host_env=None, reload=None, reset=None, **parameters)[source]

Starts and/or configure an accelerator.

Parameters:
  • stop_mode (str or int) – Host stop mode. If not None, override current “stop_mode” value. See “apyfal.host.Host.stop_mode” property for more information and possible values.
  • src (path-like object or file-like object) – Depending on the accelerator, a configuration data need to be loaded before a process can be run. Path-like object can be path, URL or cloud object URL.
  • parameters (str, path-like object or dict) – Accelerator configuration specific parameters Can also be a full configuration parameters dictionary (Or JSON equivalent as str literal or apyfal.storage URL to file) Parameters dictionary override default configuration values, individuals specific parameters overrides parameters dictionary values. Take a look to accelerator documentation for more information on possible parameters. Path-like object can be path, URL or cloud object URL.
  • reload (bool) – Force reload of FPGA bitstream.
  • reset (bool) – Force reset of FPGA logic.
  • info_dict (dict or None) – If a dict passed, this dict is updated with extra information from operation.
  • host_env (dict) – Overrides Accelerator “env”.
stop(stop_mode=None, info_dict=None)[source]

Stop accelerator session and accelerator host depending of the parameters

Parameters:
  • stop_mode (str or int) – Host stop mode. If not None, override current “stop_mode” value. See “apyfal.host.Host.stop_mode” property for more information and possible values.
  • info_dict (dict or None) – If a dict passed, this dict is updated with extra information from current operation.
class apyfal.AcceleratorPoolExecutor(accelerator=None, config=None, accelize_client_id=None, accelize_secret_id=None, host_type=None, stop_mode='term', workers_count=4, **host_kwargs)[source]

An executor that uses a pool of workers_count identically configured accelerator to execute calls asynchronously.

This class provides the full accelerator features by handling Accelerator and its host.

Parameters:
  • accelerator (str) – Name of the accelerator to initialize, to know the accelerator list please visit “https://accelstore.accelize.com”.
  • config (apyfal.configuration.Configuration, path-like object or file-like object) – If not set, will search it in current working directory, in current user “home” folder. If none found, will use default configuration values. Path-like object can be path, URL or cloud object URL.
  • accelize_client_id (str) – Accelize Client ID. Client ID is part of the access key generated on “https:/accelstore.accelize.com/user/applications”.
  • accelize_secret_id (str) – Accelize Secret ID. Secret ID come with xlz_client_id.
  • host_type (str) – Type of host to use.
  • stop_mode (str or int) – Host stop mode. Default to ‘term’ if new host, or ‘keep’ if already existing host. See “apyfal.host.Host.stop_mode” property for more information and possible values.
  • workers_count (int) – Number of accelerator workers.
  • host_kwargs – Keyword arguments related to specific host. See targeted host class to see full list of arguments.
accelerators

Accelerator workers.

Returns:Accelerators
Return type:list of apyfal.Accelerator
clients

Accelerator workers clients.

Returns:Clients
Return type:list of apyfal.client.AcceleratorClient
hosts

Accelerator workers hosts.

Returns:Hosts
Return type:list of apyfal.host.Host subclass
process_map(srcs=None, dsts=None, timeout=None, info_list=None, **parameters)

Map process execution on multiples files.

Parameters:
  • srcs (iterable of path-like object or file-like object) – Iterable of input data to process. Must be an iterable of “src” parameters of the “process” method. Path-like object can be path, URL or cloud object URL.
  • dsts (iterable of path-like object or file-like object) – Iterable of output data. Must be an iterable of “dst” parameters of the “process” method. Path-like object can be path, URL or cloud object URL.
  • timeout (float) – The maximum number of seconds to wait. If None, then there is no limit on the wait time.
  • parameters (path-like object, str or dict) – Accelerator process specific parameters Can also be a full process parameters dictionary (Or JSON equivalent as str literal) Parameters dictionary override default configuration values, individuals specific parameters overrides parameters dictionary values. Take a look to accelerator documentation for more information on possible parameters. Path-like object can be path, URL or cloud object URL.
  • info_list (list) – If a list passed, this list is updated with “info_dict” extra information dicts for each process operation.
Returns:

Results.

Return type:

generator

Raises:

concurrent.futures.TimeoutError – “timeout” reached on at least one task.

process_submit(src=None, dst=None, info_dict=None, **parameters)[source]

Schedules the process operation to be executed and returns a Future object representing the execution.

See “apyfal.Accelerator.process”.

Parameters:
  • src (path-like object or file-like object) – Source data to process. Path-like object can be path, URL or cloud object URL.
  • dst (path-like object or file-like object) – Processed data destination. Path-like object can be path, URL or cloud object URL.
  • parameters (path-like object, str or dict) – Accelerator process specific parameters Can also be a full process parameters dictionary (Or JSON equivalent as str literal) Parameters dictionary override default configuration values, individuals specific parameters overrides parameters dictionary values. Take a look to accelerator documentation for more information on possible parameters. Path-like object can be path, URL or cloud object URL.
  • info_dict (dict or None) – If a dict passed, this dict is updated with extra information from current operation. The dict will be updated on task completion.
Returns:

Future object representing execution.

See “apyfal.Accelerator.process” method for “Future.result()” content.

Return type:

concurrent.futures.Future

start(stop_mode=None, src=None, host_env=None, reload=None, reset=None, info_list=None, **parameters)[source]

Starts and/or configure all accelerators in the pool.

Parameters:
  • stop_mode (str or int) – Host stop mode. If not None, override current “stop_mode” value. See “apyfal.host.Host.stop_mode” property for more information and possible values.
  • src (path-like object or file-like object) – Depending on the accelerator, a configuration data need to be loaded before a process can be run. Path-like object can be path, URL or cloud object URL.
  • parameters (str, path-like object or dict) – Accelerator configuration specific parameters Can also be a full configuration parameters dictionary (Or JSON equivalent as str literal or apyfal.storage URL to file) Parameters dictionary override default configuration values, individuals specific parameters overrides parameters dictionary values. Take a look to accelerator documentation for more information on possible parameters. Path-like object can be path, URL or cloud object URL.
  • reload (bool) – Force reload of FPGA bitstream.
  • reset (bool) – Force reset of FPGA logic.
  • host_env (dict) – Overrides Accelerator “env”.
  • info_list (list) – If a list passed, this list is updated with “info_dict” extra information dicts for each accelerator.
Returns:

List of “Accelerator.start” results.

Return type:

list

stop(stop_mode=None, wait=True, info_list=None)[source]

Signal the executor that it should free any resources that it is using when the currently pending futures are done executing. Calls to process_submit() and process_map() made after shutdown will raise RuntimeError.

Stop each accelerator session and accelerator host depending of the parameters

Parameters:
  • stop_mode (str or int) – Host stop mode. If not None, override current “stop_mode” value. See “apyfal.host.Host.stop_mode” property for more information and possible values.
  • wait (bool) – Waits stop completion before return.
  • info_list (list) – If a list passed, this list is updated with “info_dict” extra information dicts for each accelerator.
Returns:

List of “Accelerator.stop” results if “info_dict”, else

list of Futures objects.

Return type:

list

apyfal.iter_accelerators(config=None, host_name_prefix=True, **filters)[source]

Iterates over all accelerators available on remote hosts.

Parameters:
  • config (apyfal.configuration.Configuration, path-like object or file-like object) – If not set, will search it in current working directory, in current user “home” folder. If none found, will use default configuration values. Path-like object can be path, URL or cloud object URL.
  • host_name_prefix (bool or str) – If True, use “host_name_prefix” from configuration; if False don’t filter by prefix; if str, uses this str as prefix
  • filters – Arguments names are host properties to filter, values are regular expressions.
Returns:

Accelerators generator

Return type:

generator

apyfal.get_logger(stdout=False, debug=False)[source]

Initialize logger

Parameters:
  • stdout (bool) – If True, configure logger to print on stdout, else use NullHandler
  • debug (bool) – If True, set logger level to Debug and show information useful to debug.
Returns:

logger instance

Return type:

logging.Logger