openvino.runtime.CompiledModel¶
- class openvino.runtime.CompiledModel(other: openvino._pyopenvino.CompiledModel)¶
Bases:
openvino._pyopenvino.CompiledModel
CompiledModel class.
CompiledModel represents Model that is compiled for a specific device by applying multiple optimization transformations, then mapping to compute kernels.
- __init__(self: openvino._pyopenvino.CompiledModel, other: openvino._pyopenvino.CompiledModel) None ¶
Methods
__call__
([inputs, share_inputs, ...])Callable infer wrapper for CompiledModel.
__delattr__
(name, /)Implement delattr(self, name).
__dir__
()Default dir() implementation.
__eq__
(value, /)Return self==value.
__format__
(format_spec, /)Default object formatter.
__ge__
(value, /)Return self>=value.
__getattribute__
(name, /)Return getattr(self, name).
__gt__
(value, /)Return self>value.
__hash__
()Return hash(self).
__init__
(self, other)This method is called when a class is subclassed.
__le__
(value, /)Return self<=value.
__lt__
(value, /)Return self<value.
__ne__
(value, /)Return self!=value.
__new__
(**kwargs)Helper for pickle.
__reduce_ex__
(protocol, /)Helper for pickle.
__repr__
(self)__setattr__
(name, value, /)Implement setattr(self, name, value).
Size of object in memory, in bytes.
__str__
()Return str(self).
Abstract classes can override this to customize issubclass().
Creates an inference request object used to infer the compiled model.
export_model
(*args, **kwargs)Overloaded function.
get_property
(self, property)Gets properties for current compiled model.
get_runtime_model
(self)Gets runtime model information from a device.
infer_new_request
([inputs])Infers specified input(s) in synchronous mode.
input
(*args, **kwargs)Overloaded function.
output
(*args, **kwargs)Overloaded function.
set_property
(*args, **kwargs)Overloaded function.
Attributes
Gets all inputs of a compiled model.
Gets all outputs of a compiled model.
- __call__(inputs: Optional[Any] = None, share_inputs: bool = True, share_outputs: bool = False, *, decode_strings: bool = True) openvino.runtime.utils.data_helpers.wrappers.OVDict ¶
Callable infer wrapper for CompiledModel.
Infers specified input(s) in synchronous mode.
Blocks all methods of CompiledModel while request is running.
Method creates new temporary InferRequest and run inference on it. It is advised to use a dedicated InferRequest class for performance, optimizing workflows, and creating advanced pipelines.
This method stores created InferRequest inside CompiledModel object, which can be later reused in consecutive calls.
The allowed types of keys in the inputs dictionary are:
int
str
openvino.runtime.ConstOutput
The allowed types of values in the inputs are:
numpy.ndarray and all the types that are castable to it, e.g. torch.Tensor
openvino.runtime.Tensor
Can be called with only one openvino.runtime.Tensor or numpy.ndarray, it will work only with one-input models. When model has more inputs, function throws error.
- Parameters
inputs (Any, optional) – Data to be set on input tensors.
share_inputs (bool, optional) –
Enables share_inputs mode. Controls memory usage on inference’s inputs.
If set to False inputs the data dispatcher will safely copy data to existing Tensors (including up- or down-casting according to data type, resizing of the input Tensor). Keeps Tensor inputs “as-is”.
If set to True the data dispatcher tries to provide “zero-copy” Tensors for every input in form of: * numpy.ndarray and all the types that are castable to it, e.g. torch.Tensor Data that is going to be copied: * numpy.ndarray which are not C contiguous and/or not writable (WRITEABLE flag is set to False) * inputs which data types are mismatched from Infer Request’s inputs * inputs that should be in BF16 data type * scalar inputs (i.e. np.float_/str/bytes/int/float) * lists of simple data types (i.e. str/bytes/int/float) Keeps Tensor inputs “as-is”.
Note: Use with extra care, shared data can be modified during runtime! Note: Using share_inputs may result in extra memory overhead.
Default value: True
share_outputs (bool, optional) –
Enables share_outputs mode. Controls memory usage on inference’s outputs.
If set to False outputs will safely copy data to numpy arrays.
If set to True the data will be returned in form of views of output Tensors. This mode still returns the data in format of numpy arrays but lifetime of the data is connected to OpenVINO objects.
Note: Use with extra care, shared data can be modified or lost during runtime! Note: String/textual data will always be copied!
Default value: False
decode_strings (bool, optional, keyword-only) –
Controls decoding outputs of textual based data.
If set to True string outputs will be returned as numpy arrays of U kind.
If set to False string outputs will be returned as numpy arrays of S kind.
Default value: True
- Returns
Dictionary of results from output tensors with port/int/str as keys.
- Return type
OVDict
- __class__¶
alias of
pybind11_builtins.pybind11_type
- __delattr__(name, /)¶
Implement delattr(self, name).
- __dir__()¶
Default dir() implementation.
- __eq__(value, /)¶
Return self==value.
- __format__(format_spec, /)¶
Default object formatter.
- __ge__(value, /)¶
Return self>=value.
- __getattribute__(name, /)¶
Return getattr(self, name).
- __gt__(value, /)¶
Return self>value.
- __hash__()¶
Return hash(self).
- __init__(self: openvino._pyopenvino.CompiledModel, other: openvino._pyopenvino.CompiledModel) None ¶
- __init_subclass__()¶
This method is called when a class is subclassed.
The default implementation does nothing. It may be overridden to extend subclasses.
- __le__(value, /)¶
Return self<=value.
- __lt__(value, /)¶
Return self<value.
- __ne__(value, /)¶
Return self!=value.
- __new__(**kwargs)¶
- __reduce__()¶
Helper for pickle.
- __reduce_ex__(protocol, /)¶
Helper for pickle.
- __repr__(self: openvino._pyopenvino.CompiledModel) str ¶
- __setattr__(name, value, /)¶
Implement setattr(self, name, value).
- __sizeof__()¶
Size of object in memory, in bytes.
- __str__()¶
Return str(self).
- __subclasshook__()¶
Abstract classes can override this to customize issubclass().
This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).
- create_infer_request() openvino.runtime.ie_api.InferRequest ¶
Creates an inference request object used to infer the compiled model.
The created request has allocated input and output tensors.
- Returns
New InferRequest object.
- Return type
- export_model(*args, **kwargs)¶
Overloaded function.
export_model(self: openvino._pyopenvino.CompiledModel) -> bytes
Exports the compiled model to bytes/output stream.
GIL is released while running this function.
- return
Bytes object that contains this compiled model.
- rtype
bytes
user_stream = compiled.export_model() with open('./my_model', 'wb') as f: f.write(user_stream) # ... new_compiled = core.import_model(user_stream, "CPU")
export_model(self: openvino._pyopenvino.CompiledModel, model_stream: object) -> None
Exports the compiled model to bytes/output stream.
Advanced version of export_model. It utilizes, streams from the standard Python library io.
Function performs flushing of the stream, writes to it, and then rewinds the stream to the beginning (using seek(0)).
GIL is released while running this function.
- param model_stream
A stream object to which the model will be serialized.
- type model_stream
io.BytesIO
- rtype
None
user_stream = io.BytesIO() compiled.export_model(user_stream) with open('./my_model', 'wb') as f: f.write(user_stream.getvalue()) # or read() if seek(0) was applied before # ... new_compiled = core.import_model(user_stream, "CPU")
- get_property(self: openvino._pyopenvino.CompiledModel, property: str) object ¶
Gets properties for current compiled model.
- Parameters
name (str) – Property name.
- Return type
Any
- get_runtime_model(self: openvino._pyopenvino.CompiledModel) openvino._pyopenvino.Model ¶
Gets runtime model information from a device.
This object (returned model) represents the internal device-specific model which is optimized for the particular accelerator. It contains device-specific nodes, runtime information, and can be used only to understand how the source model is optimized and which kernels, element types, and layouts are selected.
- Returns
Model, containing Executable Graph information.
- Return type
- infer_new_request(inputs: Optional[Any] = None) openvino.runtime.utils.data_helpers.wrappers.OVDict ¶
Infers specified input(s) in synchronous mode.
Blocks all methods of CompiledModel while request is running.
Method creates new temporary InferRequest and run inference on it. It is advised to use a dedicated InferRequest class for performance, optimizing workflows, and creating advanced pipelines.
The allowed types of keys in the inputs dictionary are:
int
str
openvino.runtime.ConstOutput
The allowed types of values in the inputs are:
numpy.ndarray and all the types that are castable to it, e.g. torch.Tensor
openvino.runtime.Tensor
Can be called with only one openvino.runtime.Tensor or numpy.ndarray, it will work only with one-input models. When model has more inputs, function throws error.
- Parameters
inputs (Any, optional) – Data to be set on input tensors.
- Returns
Dictionary of results from output tensors with port/int/str keys.
- Return type
OVDict
- input(*args, **kwargs)¶
Overloaded function.
input(self: openvino._pyopenvino.CompiledModel) -> openvino._pyopenvino.ConstOutput
Gets a single input of a compiled model. If a model has more than one input, this method throws an exception.
- return
A compiled model input.
- rtype
openvino.runtime.ConstOutput
input(self: openvino._pyopenvino.CompiledModel, index: int) -> openvino._pyopenvino.ConstOutput
Gets input of a compiled model identified by an index. If the input with given index is not found, this method throws an exception.
- param index
An input index.
- type index
int
- return
A compiled model input.
- rtype
openvino.runtime.ConstOutput
input(self: openvino._pyopenvino.CompiledModel, tensor_name: str) -> openvino._pyopenvino.ConstOutput
Gets input of a compiled model identified by a tensor_name. If the input with given tensor name is not found, this method throws an exception.
- param tensor_name
An input tensor name.
- type tensor_name
str
- return
A compiled model input.
- rtype
openvino.runtime.ConstOutput
- property inputs¶
Gets all inputs of a compiled model.
- Returns
Inputs of a compiled model.
- Return type
- output(*args, **kwargs)¶
Overloaded function.
output(self: openvino._pyopenvino.CompiledModel) -> openvino._pyopenvino.ConstOutput
Gets a single output of a compiled model. If the model has more than one output, this method throws an exception.
- return
A compiled model output.
- rtype
openvino.runtime.ConstOutput
output(self: openvino._pyopenvino.CompiledModel, index: int) -> openvino._pyopenvino.ConstOutput
Gets output of a compiled model identified by an index. If the output with given index is not found, this method throws an exception.
- param index
An output index.
- type index
int
- return
A compiled model output.
- rtype
openvino.runtime.ConstOutput
output(self: openvino._pyopenvino.CompiledModel, tensor_name: str) -> openvino._pyopenvino.ConstOutput
Gets output of a compiled model identified by a tensor_name. If the output with given tensor name is not found, this method throws an exception.
- param tensor_name
An output tensor name.
- type tensor_name
str
- return
A compiled model output.
- rtype
openvino.runtime.ConstOutput
- property outputs¶
Gets all outputs of a compiled model.
- Returns
Outputs of a compiled model.
- Return type
- set_property(*args, **kwargs)¶
Overloaded function.
set_property(self: openvino._pyopenvino.CompiledModel, properties: Dict[str, object]) -> None
Sets properties for current compiled model.
- param properties
Dict of pairs: (property name, property value)
- type properties
dict
- rtype
None
set_property(self: openvino._pyopenvino.CompiledModel, property: Tuple[str, object]) -> None
Sets properties for current compiled model.
- param property
Tuple of (property name, matching property value).
- type property
tuple