Group Basics¶
- group ov_model_cpp_api
OpenVINO Core C++ API to work with ov::Model, dynamic and static shapes, types
Functions
-
template<typename ForwardIt>
size_t shape_size(ForwardIt start_dim, const ForwardIt end_dim)¶ Number of elements in a subset of dimensions of a shape. Returns a product of dimensions in a range [start_dim;end_dim)
-
template<typename SHAPE_TYPE>
size_t shape_size(const SHAPE_TYPE &shape)¶ Number of elements in spanned by a shape.
-
class Dimension
- #include <dimension.hpp>
Class representing a dimension, which may be dynamic (undetermined until runtime), in a shape or shape-like object.
Static dimensions may be implicitly converted from value_type. A dynamic dimension is constructed with Dimension() or Dimension::dynamic().
Public Functions
-
Dimension(value_type dimension)
Construct a static dimension.
- Parameters
dimension – Value of the dimension.
-
Dimension(value_type min_dimension, value_type max_dimension)
Construct a dynamic dimension with bounded range.
- Parameters
min_dimension – The lower inclusive limit for the dimension
max_dimension – The upper inclusive limit for the dimension
-
Dimension(const std::string &str)
Construct a dimension from string.
- Parameters
str – String to parse to dimension.
-
Dimension() = default
Construct a dynamic dimension with range [0, …].
-
inline bool is_static() const
Check whether this dimension is static.
- Returns
true
if the dimension is static, elsefalse
.
-
inline bool is_dynamic() const
Check whether this dimension is dynamic.
- Returns
false
if the dimension is static, elsetrue
.
-
value_type get_length() const
Convert this dimension to
value_type
. This dimension must be static and non-negative.- Throws
std::invalid_argument – If this dimension is dynamic or negative.
-
inline const Interval &get_interval() const
Return the interval of valid lengths.
-
bool same_scheme(const Dimension &dim) const
Check whether this dimension represents the same scheme as the argument (both dynamic, or equal).
- Parameters
dim – The other dimension to compare this dimension to.
- Returns
true
if this dimension anddim
are both dynamic, or if they are both static and equal; otherwise,false
.
-
bool compatible(const Dimension &d) const
Check whether this dimension is capable of being merged with the argument dimension.
Two dimensions are considered compatible if it is possible to merge them. (See Dimension::merge.)
- Parameters
d – The dimension to compare this dimension with.
- Returns
true
if this dimension is compatible withd
, elsefalse
.
-
bool relaxes(const Dimension &d) const
Check whether this dimension is a relaxation of the argument.
A dimension
d1
relaxes (or is a relaxation of)d2
ifd1
andd2
are static and equal, ord1
is dynamic.d1.relaxes(d2)
is equivalent tod2.refines(d1)
.- Parameters
d – The dimension to compare this dimension with.
- Returns
true
if this dimension relaxesd
, elsefalse
.
-
bool refines(const Dimension &d) const
Check whether this dimension is a refinement of the argument.
A dimension
d2
refines (or is a refinement of)d1
ifd1
andd2
are static and equal, ord2
is dynamic.d1.refines(d2)
is equivalent tod2.relaxes(d1)
.- Parameters
d – The dimension to compare this dimension with.
- Returns
true
if this dimension relaxesd
, elsefalse
.
-
Dimension operator+(const Dimension &dim) const
Addition operator for Dimension.
- Parameters
dim – Right operand for addition.
- Returns
Smallest interval dimension enclosing inputs
-
Dimension operator-(const Dimension &dim) const
Subtraction operator for Dimension.
- Parameters
dim – Right operand for subtraction.
- Returns
Smallest interval dimension enclosing inputs
-
Dimension operator/(const value_type divisor) const
Division operator for Dimension divided by a value_type parameter.
- Parameters
divisor – Right operand for division.
- Returns
Smallest interval dimension enclosing inputs
-
inline Dimension &operator/=(const value_type divisor)
Divided-into operator for Dimension.
- Parameters
divisor – Right operand for multiplication.
- Returns
A reference to
*this
, after updating*this
to the value*this * dim
.
-
Dimension operator*(const Dimension &dim) const
Multiplication operator for Dimension.
- Parameters
dim – Right operand for multiplicaiton.
- Returns
Smallest interval containing all “produces” which are 0 if either of
this
ordim
has length0
, else unbounded if either is unbounded, else product of lengths.
-
inline Dimension &operator+=(const Dimension &dim)
Add-into operator for Dimension.
- Parameters
dim – Right operand for addition.
- Returns
A reference to
*this
, after updating*this
to the value*this + dim
.
-
inline Dimension &operator*=(const Dimension &dim)
Multiply-into operator for Dimension.
- Parameters
dim – Right operand for multiplication.
- Returns
A reference to
*this
, after updating*this
to the value*this * dim
.
-
std::string to_string() const
String representation of Dimension.
Public Static Functions
-
static bool merge(Dimension &dst, const Dimension &d1, const Dimension &d2)
Try to merge two Dimension objects together.
If
d1
is dynamic, writesd2
todst
and returnstrue
.If
d2
is dynamic, writesd1
todst
and returnstrue
.If
d1
andd2
are static and equal, writesd1
todst
and returnstrue
.If
d1
andd2
are both static and unequal, leavesdst
unchanged and returnsfalse
.
- Parameters
dst – [out] Reference to write the merged Dimension into.
d1 – First dimension to merge.
d2 – Second dimension to merge.
- Returns
true
if merging succeeds, elsefalse
.
-
static bool broadcast_merge(Dimension &dst, const Dimension &d1, const Dimension &d2)
Try to merge two Dimension objects together with implicit broadcasting of unit-sized dimension to non unit-sized dimension.
-
static inline Dimension dynamic()
Create a dynamic dimension.
- Returns
A dynamic dimension.
-
Dimension(value_type dimension)
-
class Extension
- #include <extension.hpp>
The class provides the base interface for OpenVINO extensions.
Subclassed by ov::BaseOpExtension, ov::frontend::ConversionExtensionBase, ov::frontend::DecoderTransformationExtension, ov::frontend::ProgressReporterExtension, ov::frontend::TelemetryExtension
-
class Model : public std::enable_shared_from_this<Model>
- #include <model.hpp>
A user-defined model.
Public Functions
-
explicit Model(const ov::OutputVector &results, const std::string &name = "")
Constructs a Model. Lists of parameters and variables will be generated automatically based on traversing the graph from the results.
-
Model(const ov::OutputVector &results, const ov::SinkVector &sinks, const std::string &name = "")
Constructs a Model. Lists of parameters and variables will be generated automatically based on traversing the graph from the results and the sinks.
-
size_t get_output_size() const
Return the number of outputs for this Model.
-
const ov::element::Type &get_output_element_type(size_t i) const
Return the element type of output i.
-
const Shape &get_output_shape(size_t i) const
Return the shape of element i.
-
const PartialShape &get_output_partial_shape(size_t i) const
Return the partial shape of element i.
-
const std::string &get_name() const
Get the unique name of the model.
- Returns
A const reference to the model’s unique name.
-
void set_friendly_name(const std::string &name)
Sets a friendly name for a model. This does not overwrite the unique name of the model and is retrieved via get_friendly_name(). Used mainly for debugging.
- Parameters
name – is the friendly name to set
-
const std::string &get_friendly_name() const
Gets the friendly name for a model. If no friendly name has been set via set_friendly_name then the model’s unique name is returned.
- Returns
A const reference to the model’s friendly name.
-
size_t get_graph_size() const
Returns the sum of the size of all nodes in the graph plus the size of all constant data. This has little value beyond comparing the relative size of graphs and should not be considered the actual memory consumption of a graph.
-
bool is_dynamic() const
Returns true if any of the op’s defined in the model contains partial shape.
Replace the
parameter_index
th parameter of the model withparameter
.All users of the
parameter_index
th parameter are redirected toparameter
, and theparameter_index
th entry in the model parameter list is replaced withparameter
.- Parameters
parameter_index – The index of the parameter to replace.
parameter – The parameter to substitute for the
parameter_index
th parameter.
-
inline const ov::ParameterVector &get_parameters() const
Return the model parameters.
-
inline const ov::ResultVector &get_results() const
Return a list of model’s outputs.
Index for parameter, or -1.
-
int64_t get_result_index(const ov::Output<ov::Node> &value) const
Return the index of this model’s Result represented by the “value” Output object. This method returns -1 if an the passed output is not related to the Results of a model.
-
int64_t get_result_index(const ov::Output<const ov::Node> &value) const
Return the index of this model’s Result represented by the “value” Output object. This method returns -1 if an the passed output is not related to the Results of a model.
-
bool evaluate(ov::TensorVector &output_tensors, const ov::TensorVector &input_tensors, ov::EvaluationContext &evaluation_context) const
Evaluate the model on inputs, putting results in outputs.
- Parameters
output_tensors – Tensors for the outputs to compute. One for each result
input_tensors – Tensors for the inputs. One for each inputs.
evaluation_context – Storage of additional settings and attributes that can be used when evaluating the model. This additional information can be shared across nodes.
-
bool evaluate(ov::TensorVector &output_tensors, const ov::TensorVector &input_tensors) const
Evaluate the model on inputs, putting results in outputs.
- Parameters
output_tensors – Tensors for the outputs to compute. One for each result
input_tensors – Tensors for the inputs. One for each inputs.
-
inline const ov::SinkVector &get_sinks() const
Return a list of model’s sinks.
-
void add_sinks(const ov::SinkVector &sinks)
Add new sink nodes to the list. Method doesn’t validate graph, it should be done manually after all changes.
- Parameters
sinks – new sink nodes
Delete sink node from the list of sinks. Method doesn’t delete node from graph.
- Parameters
sink – Sink to delete
-
void add_results(const ov::ResultVector &results)
Add new Result nodes to the list. Method doesn’t validate graph, it should be done manually after all changes.
- Parameters
results – new Result nodes
Delete Result node from the list of results. Method will not delete node from graph.
- Parameters
result – Result node to delete
-
void add_parameters(const ov::ParameterVector ¶ms)
Add new Parameter nodes to the list.
Method doesn’t change or validate graph, it should be done manually. For example, if you want to replace
ReadValue
node byParameter
, you should do the following steps:replace node
ReadValue
byParameter
in graphcall add_parameter() to add new input to the list
call graph validation to check correctness of changes
- Parameters
params – new Parameter nodes
Delete Parameter node from the list of parameters. Method will not delete node from graph. You need to replace Parameter with other operation manually. Attention: Indexing of parameters can be changed.
Possible use of method is to replace input by variable. For it the following steps should be done:
Parameter
node should be replaced byReadValue
call remove_parameter(param) to remove input from the list
check if any parameter indexes are saved/used somewhere, update it for all inputs because indexes can be changed
call graph validation to check all changes
- Parameters
param – Parameter node to delete
-
void add_variables(const ov::op::util::VariableVector &variables)
Add new variables to the list. Method doesn’t validate graph, it should be done manually after all changes.
- Parameters
variables – new variables to add
-
void remove_variable(const ov::op::util::Variable::Ptr &variable)
Delete variable from the list of variables. Method doesn’t delete nodes that used this variable from the graph.
- Parameters
variable – Variable to delete
-
inline const ov::op::util::VariableVector &get_variables() const
Return a list of model’s variables.
-
ov::op::util::Variable::Ptr get_variable_by_id(const std::string &variable_id) const
Return a variable by specified variable_id.
-
inline RTMap &get_rt_info()
Returns a runtime info.
- Returns
reference to ov::AnyMap with runtime info
-
inline const RTMap &get_rt_info() const
Returns a constant runtime info.
- Returns
reference to const ov::AnyMap with runtime info
-
template<class T, class ...Args, typename std::enable_if<!std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(Args... args) const Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.
- Template Parameters
T – the type of returned value
Args – types of variadic arguments
- Parameters
args – path to the runtime attribute
- Returns
constant reference to value from runtime info
-
template<class T, class ...Args, typename std::enable_if<std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(Args... args) const Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.
- Template Parameters
T – the type of returned value
Args – types of variadic arguments
- Parameters
args – path to the runtime attribute
- Returns
constant reference to value from runtime info
-
template<class T, typename std::enable_if<!std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(const std::vector<std::string> &args) const Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.
- Template Parameters
T – the type of returned value
- Parameters
args – vector with path to the runtime attribute
- Returns
constant reference to value from runtime info
-
template<class T, typename std::enable_if<std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(const std::vector<std::string> &args) const Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.
- Template Parameters
T – the type of returned value
- Parameters
args – vector with path to the runtime attribute
- Returns
constant reference to value from runtime info
-
template<class ...Args>
inline bool has_rt_info(Args... args) const Checks if given path exists in runtime info.
- Template Parameters
Args – types of variadic arguments
- Parameters
args – path to the runtime attribute
- Returns
true if path exists, otherwise false
-
bool has_rt_info(const std::vector<std::string> &args) const
Checks if given path exists in runtime info.
- Parameters
args – vector with path to the runtime attribute
- Returns
true if path exists, otherwise false
-
template<class T, class ...Args>
inline void set_rt_info(const T &argument, Args... args) Add value inside the runtime info.
- Template Parameters
T – type of new value
Args – types of variadic arguments
- Parameters
argument – value for the runtime info
args – path to the runtime attribute
-
template<class T>
inline void set_rt_info(const T &argument, const std::vector<std::string> &args) Add value inside the runtime info.
- Template Parameters
T – type of new value
- Parameters
argument – value for the runtime info
args – vector with path to the runtime attribute
-
explicit Model(const ov::OutputVector &results, const std::string &name = "")
-
class Node : public std::enable_shared_from_this<Node>
- #include <node.hpp>
Nodes are the backbone of the graph of Value dataflow. Every node has zero or more nodes as arguments and one value, which is either a tensor or a (possibly empty) tuple of values.
Subclassed by ov::op::Op, ov::pass::pattern::op::Pattern
Public Functions
-
virtual void validate_and_infer_types()
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual const ov::op::AutoBroadcastSpec &get_autob() const
- Returns
the autobroadcasr spec
-
virtual bool has_evaluate() const
Allows to get information about availability of evaluate method for the current operation.
-
virtual bool evaluate(ov::TensorVector &output_values, const ov::TensorVector &input_values) const
Evaluates the op on input_values putting results in output_values.
- Parameters
output_values – Tensors for the outputs to compute. One for each result
input_values – Tensors for the inputs. One for each inputs.
- Returns
true if successful
-
virtual bool evaluate(ov::TensorVector &output_values, const ov::TensorVector &input_values, const ov::EvaluationContext &evaluationContext) const
Evaluates the op on input_values putting results in output_values.
- Parameters
output_values – Tensors for the outputs to compute. One for each result
input_values – Tensors for the inputs. One for each inputs.
evaluation_context – Storage of additional settings and attributes that can be used when evaluating the op.
- Returns
true if successful
-
inline virtual OutputVector decompose_op() const
Decomposes the FusedOp into a sub-graph consisting of core openvino ops.
- Returns
A vector of nodes comprising the sub-graph. The order of output tensors must match the match output tensors of the FusedOp
-
virtual const type_info_t &get_type_info() const = 0
Returns the NodeTypeInfo for the node’s class. During transition to type_info, returns a dummy type_info for Node if the class has not been updated yet.
-
void set_arguments(const NodeVector &arguments)
Sets/replaces the arguments with new arguments.
-
void set_arguments(const OutputVector &arguments)
Sets/replaces the arguments with new arguments.
-
void set_argument(size_t position, const Output<Node> &argument)
Sets/replaces the arguments with new arguments.
-
void set_output_size(size_t output_size)
Sets the number of outputs.
-
virtual std::string description() const
Get the string name for the type of the node, such as
Add
orMultiply
. The class name, must not contain spaces as it is used for codegen.- Returns
A const reference to the node’s type name
-
const std::string &get_name() const
Get the unique name of the node.
- Returns
A const reference to the node’s unique name.
-
void set_friendly_name(const std::string &name)
Sets a friendly name for a node. This does not overwrite the unique name of the node and is retrieved via get_friendly_name(). Used mainly for debugging. The friendly name may be set exactly once.
- Parameters
name – is the friendly name to set
-
const std::string &get_friendly_name() const
Gets the friendly name for a node. If no friendly name has been set via set_friendly_name then the node’s unique name is returned.
- Returns
A const reference to the node’s friendly name.
-
virtual std::ostream &write_description(std::ostream &os, uint32_t depth = 0) const
Writes a description of a node to a stream.
- Parameters
os – The stream; should be returned
depth – How many levels of inputs to describe
- Returns
The stream os
-
const std::vector<std::shared_ptr<Node>> &get_control_dependencies() const
Get control dependencies registered on the node.
-
const std::vector<Node*> &get_control_dependents() const
Get nodes dependent on this node.
This node cannot execute until node executes.
Remove the dependency of this node on node.
-
void clear_control_dependencies()
Remove all dependencies from this node.
-
void clear_control_dependents()
Remove this node as a dependency from all dependent nodes.
This node absorbs the control dependencies of source_node.
This node becomes a dependent of every node dependent on source_node.
This node’s control dependencies are replaced by replacement.
-
size_t get_output_size() const
Returns the number of outputs from the node.
-
const element::Type &get_element_type() const
Checks that there is exactly one output and returns its element type.
-
const Shape &get_output_shape(size_t i) const
Returns the shape for output i.
-
const PartialShape &get_output_partial_shape(size_t i) const
Returns the partial shape for output i.
-
Output<const Node> get_default_output() const
Return the output to use when converting to an Output<Node> with no index specified. Throws when not supported.
-
virtual size_t get_default_output_index() const
Returns the output of the default output, or throws if there is none.
-
size_t no_default_index() const
Throws no default.
-
const Shape &get_shape() const
Checks that there is exactly one output and returns its shape.
-
descriptor::Tensor &get_output_tensor(size_t i) const
Returns the tensor for output or input i.
-
size_t get_input_size() const
Returns the number of inputs for the op.
-
const Shape &get_input_shape(size_t i) const
Returns the shape of input i.
-
const PartialShape &get_input_partial_shape(size_t i) const
Returns the partial shape of input i.
True if this and node have one output with same element type and shape.
-
NodeVector get_users(bool check_is_used = false) const
Get all the nodes that uses the current node.
-
inline bool operator<(const Node &other) const
Use instance ids for comparison instead of memory addresses to improve determinism.
-
std::vector<Input<Node>> inputs()
- Returns
A vector containing a handle for each of this node’s inputs, in order.
-
std::vector<Input<const Node>> inputs() const
- Returns
A vector containing a handle for each of this node’s inputs, in order.
-
std::vector<Output<Node>> input_values() const
- Returns
A vector containing the values for each input
-
std::vector<Output<Node>> outputs()
- Returns
A vector containing a handle for each of this node’s outputs, in order.
-
std::vector<Output<const Node>> outputs() const
- Returns
A vector containing a handle for each of this node’s outputs, in order.
-
Input<Node> input(size_t input_index)
- Throws
std::out_of_range – if the node does not have at least
input_index+1
inputs.- Returns
A handle to the
input_index
th input of this node.
-
Input<const Node> input(size_t input_index) const
- Throws
std::out_of_range – if the node does not have at least
input_index+1
inputs.- Returns
A handle to the
input_index
th input of this node.
-
virtual void validate_and_infer_types()
-
template<>
class Input<Node> - #include <node_input.hpp>
A handle for one of a node’s inputs.
Public Functions
-
Input(Node *node, size_t index)
Constructs a Input.
- Parameters
node – Pointer to the node for the input handle.
index – The index of the input.
-
Node *get_node() const
- Returns
A pointer to the node referenced by this input handle.
-
size_t get_index() const
- Returns
The index of the input referred to by this input handle.
-
const element::Type &get_element_type() const
- Returns
The element type of the input referred to by this input handle.
-
const Shape &get_shape() const
- Returns
The shape of the input referred to by this input handle.
-
const PartialShape &get_partial_shape() const
- Returns
The partial shape of the input referred to by this input handle.
-
Output<Node> get_source_output() const
- Returns
A handle to the output that is connected to this input.
-
descriptor::Tensor &get_tensor() const
- Returns
A reference to the tensor descriptor for this input.
-
std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
- Returns
A shared pointer to the tensor descriptor for this input.
-
bool get_is_relevant_to_shapes() const
- Returns
true if this input is relevant to its node’s output shapes; else false.
-
bool get_is_relevant_to_values() const
- Returns
true if this input is relevant to its node’s output values; else false.
-
void replace_source_output(const Output<Node> &new_source_output) const
Replaces the source output of this input.
- Parameters
new_source_output – A handle for the output that will replace this input’s source.
-
RTMap &get_rt_info()
- Returns
The reference to runtime info map
-
const RTMap &get_rt_info() const
- Returns
The constant reference to runtime info map
-
Input(Node *node, size_t index)
-
template<>
class Input<const Node> - #include <node_input.hpp>
A handle for one of a node’s inputs.
Public Functions
-
Input(const Node *node, size_t index)
Constructs a Input.
- Parameters
node – Pointer to the node for the input handle.
index – The index of the input.
-
const Node *get_node() const
- Returns
A pointer to the node referenced by this input handle.
-
size_t get_index() const
- Returns
The index of the input referred to by this input handle.
-
const element::Type &get_element_type() const
- Returns
The element type of the input referred to by this input handle.
-
const Shape &get_shape() const
- Returns
The shape of the input referred to by this input handle.
-
const PartialShape &get_partial_shape() const
- Returns
The partial shape of the input referred to by this input handle.
-
Output<Node> get_source_output() const
- Returns
A handle to the output that is connected to this input.
-
descriptor::Tensor &get_tensor() const
- Returns
A reference to the tensor descriptor for this input.
-
std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
- Returns
A shared pointer to the tensor descriptor for this input.
-
bool get_is_relevant_to_shapes() const
- Returns
true if this input is relevant to its node’s output shapes; else false.
-
bool get_is_relevant_to_values() const
- Returns
true if this input is relevant to its node’s output values; else false.
-
const RTMap &get_rt_info() const
- Returns
The constant reference to runtime info map
-
Input(const Node *node, size_t index)
-
template<>
class Output<Node> - #include <node_output.hpp>
A handle for one of a node’s outputs.
Public Functions
-
Output(Node *node, size_t index)
Constructs a Output.
- Parameters
node – A pointer to the node for the output handle.
index – The index of the output.
Constructs a Output.
- Parameters
node – A
shared_ptr
to the node for the output handle.index – The index of the output.
Constructs a Output, referencing the zeroth output of the node.
- Parameters
node – A
shared_ptr
to the node for the output handle.
-
Output() = default
A null output.
-
Node *get_node() const
- Returns
A pointer to the node referred to by this output handle.
- Returns
A
shared_ptr
to the node referred to by this output handle.
-
size_t get_index() const
- Returns
The index of the output referred to by this output handle.
-
descriptor::Tensor &get_tensor() const
- Returns
A reference to the tensor descriptor for this output.
-
std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
- Returns
A shared point to the tensor ptr for this output.
- Returns
Set new tensor desc shared pointer to this output
-
const element::Type &get_element_type() const
- Returns
The element type of the output referred to by this output handle.
-
const Shape &get_shape() const
- Returns
The shape of the output referred to by this output handle.
-
const PartialShape &get_partial_shape() const
- Returns
The partial shape of the output referred to by this output handle.
-
RTMap &get_rt_info()
- Returns
The reference to runtime info map
-
const RTMap &get_rt_info() const
- Returns
The constant reference to runtime info map
-
const std::unordered_set<std::string> &get_names() const
- Returns
The tensor names associated with this output
-
std::string get_any_name() const
- Returns
Any tensor names associated with this output
-
void set_names(const std::unordered_set<std::string> &names)
- Returns
Set tensor names associated with this output
-
void add_names(const std::unordered_set<std::string> &names)
- Returns
Add tensor names associated with this output
-
std::set<Input<Node>> get_target_inputs() const
- Returns
A set containing handles for all inputs targeted by the output referenced by this output handle.
-
Output(Node *node, size_t index)
-
template<>
class Output<const Node> - #include <node_output.hpp>
A handle for one of a node’s outputs.
Public Functions
-
Output(const Node *node, size_t index)
Constructs a Output.
- Parameters
node – A pointer to the node for the output handle.
index – The index of the output.
Constructs a Output.
- Parameters
node – A
shared_ptr
to the node for the output handle.index – The index of the output.
Constructs a Output, referencing the zeroth output of the node.
- Parameters
node – A
shared_ptr
to the node for the output handle.
-
Output() = default
A null output.
-
const Node *get_node() const
- Returns
A pointer to the node referred to by this output handle.
- Returns
A
shared_ptr
to the node referred to by this output handle.
-
size_t get_index() const
- Returns
The index of the output referred to by this output handle.
-
descriptor::Tensor &get_tensor() const
- Returns
A reference to the tensor descriptor for this output.
-
std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
- Returns
A shared point to the tensor ptr for this output.
-
const element::Type &get_element_type() const
- Returns
The element type of the output referred to by this output handle.
-
const Shape &get_shape() const
- Returns
The shape of the output referred to by this output handle.
-
const PartialShape &get_partial_shape() const
- Returns
The partial shape of the output referred to by this output handle.
-
const RTMap &get_rt_info() const
- Returns
The constant reference to runtime info map
-
const std::unordered_set<std::string> &get_names() const
- Returns
The tensor names associated with this output
-
std::string get_any_name() const
- Returns
Any tensor name associated with this output
-
Output(const Node *node, size_t index)
-
class PartialShape
- #include <partial_shape.hpp>
Class representing a shape that may be partially or totally dynamic.
A PartialShape may have:
Dynamic rank. (Informal notation:
?
)Static rank, but dynamic dimensions on some or all axes. (Informal notation examples:
{1,2,?,4}
,{?,?,?}
)Static rank, and static dimensions on all axes. (Informal notation examples:
{1,2,3,4}
,{6}
,{}
)
Public Functions
-
PartialShape(std::initializer_list<Dimension> init)
Constructs a shape with static rank from an initializer list of Dimension.
Examples:
PartialShape s{2,3,4}; // rank=3, all dimensions static PartialShape s{}; // rank=0 PartialShape s{2,Dimension::dynamic(),3}; // rank=3, dimension 1 dynamic
- Parameters
init – The Dimension values for the constructed shape.
-
PartialShape(std::vector<Dimension> dimensions)
Constructs a PartialShape with static rank from a vector of Dimension.
- Parameters
dimensions – The Dimension values for the constructed shape.
-
PartialShape(const std::vector<Dimension::value_type> &dimensions)
Constructs a PartialShape with static rank from a vector of dimensions values.
- Parameters
dimensions – The Dimension values for the constructed shape.
-
PartialShape()
Constructs a static PartialShape with zero rank (the shape of a scalar).
-
PartialShape(const Shape &shape)
Constructs a static PartialShape from a PartialShape.
- Parameters
shape – The PartialShape to convert into PartialShape.
-
PartialShape(const std::string &shape)
Constructs a static PartialShape from a string.
- Parameters
shape – The string to parse into PartialShape.
-
bool is_static() const
Check if this shape is static.
A shape is considered static if it has static rank, and all dimensions of the shape are static.
- Returns
true
if this shape is static, elsefalse
.
-
inline bool is_dynamic() const
Check if this shape is dynamic.
A shape is considered static if it has static rank, and all dimensions of the shape are static.
- Returns
false
if this shape is static, elsetrue
.
-
inline Rank rank() const
Get the rank of the shape.
- Returns
The rank of the shape. This will be Rank::dynamic() if the rank of the shape is dynamic.
-
bool compatible(const PartialShape &s) const
Check whether this shape is compatible with the argument, i.e., whether it is possible to merge them.
Two shapes are compatible if
one or both of them has dynamic rank, or
both shapes have dynamic and equal rank, and their dimensions are elementwise compatible (see Dimension::compatible()).
- Parameters
s – The shape to be checked for compatibility with this shape.
- Returns
true
if this shape is compatible withs
, elsefalse
.
-
bool same_scheme(const PartialShape &s) const
Check whether this shape represents the same scheme as the argument.
Two shapes
s1
ands2
represent the same scheme ifthey both have dynamic rank, or
they both have static and equal rank
r
, and for everyi
from0
tor-1
,s1[i]
represents the same scheme ass2[i]
(see Dimension::same_scheme()).
- Parameters
s – The shape whose scheme is being compared with this shape.
- Returns
true
if this shape represents the same scheme ass
, elsefalse
.
-
bool relaxes(const PartialShape &s) const
Check whether this shape is a relaxation of the argument.
Intuitively, a PartialShape
s1
is said to relaxs2
(or is a relaxation ofs2
) if it is “more permissive” thans2
. In other words,s1
is a relaxation ofs2
if anything you can form by plugging things into the dynamic dimensions ofs2
is also something you can form by plugging things into the dynamic dimensions ofs1
, but not necessarily the other way around.s1.relaxes(s2)
is equivalent tos2.refines(s1)
.Formally, PartialShape
s1
is said to relax PartialShapes2
if:For every
i
from0
tor-1
, eithers1[i]
contains s2[i].
- Parameters
s – The shape which is being compared against this shape.
- Returns
true
if this shape relaxess
, elsefalse
.
-
bool refines(const PartialShape &s) const
Check whether this shape is a refinement of the argument.
Intuitively, a PartialShape
s1
is said to relaxs2
(or is a relaxation ofs2
) if it is “less permissive” thans2
. In other words,s1
is a relaxation ofs2
if anything you can form by plugging things into the dynamic dimensions ofs1
is also something you can form by plugging things into the dynamic dimensions ofs2
, but not necessarily the other way around.s1.refines(s2)
is equivalent tos2.relaxes(s1)
.Formally, PartialShape
s1
is said to refine PartialShapes2
if:s2
has dynamic rank, ors1
ands2
both have static rankr
, and for everyi
from0
tor-1
, eithers2[i]
is dynamic, ors1[i]
==s2[i]
.
- Parameters
s – The shape which is being compared against this shape.
- Returns
true
if this shape refiness
, elsefalse
.
-
bool merge_rank(const Rank &r)
Checks that this shape’s rank is compatible with
r
, and, if this shape’s rank is dynamic andr
is static, updates this shape to have a rank ofr
with dimensions all dynamic.- Returns
true
if this shape’s rank is compatible withr
, elsefalse
.
-
Shape to_shape() const
Convert a static PartialShape to a PartialShape.
- Throws
std::invalid_argument – If this PartialShape is dynamic.
- Returns
A new PartialShape
s
wheres[i] = size_t((*this)[i])
.
-
bool all_non_negative() const
Returns
true
if all static dimensions of the tensor are non-negative, elsefalse
.
-
Dimension &operator[](std::ptrdiff_t i)
Index operator for PartialShape, with bound checking.
- Parameters
i – The index of the dimension being selected in range [-rank, rank).
- Returns
A reference to the
i
th Dimension of this shape.
-
const Dimension &operator[](std::ptrdiff_t i) const
Index operator for PartialShape, with bound checking.
- Parameters
i – The index of the dimension being selected in range [-rank, rank).
- Returns
A reference to the
i
th Dimension of this shape.
-
inline explicit operator std::vector<Dimension>() const
Returns a vector of the dimensions. This has no meaning if dynamic.
-
Shape get_max_shape() const
Get the max bounding shape.
-
Shape get_min_shape() const
Get the min bounding shape.
-
Shape get_shape() const
Get the unique shape.
-
inline iterator begin() noexcept
Returns a read/write iterator that points to the first element in the shape. Iteration is done in ordinary element order.
-
inline const_iterator begin() const noexcept
Returns a read-only (constant) iterator that points to the first element in the shape. Iteration is done in ordinary element order.
-
inline iterator end() noexcept
Returns a read/write iterator that points one past the last element in the shape. Iteration is done in ordinary element order.
-
inline const_iterator end() const noexcept
Returns a read-only (constant) iterator that points one past the last element in the shape. Iteration is done in ordinary element order.
-
inline reverse_iterator rbegin() noexcept
Returns a read/write reverse iterator that points to the last element in the shape. Iteration is done in reverse element order.
-
inline const_reverse_iterator rbegin() const noexcept
Returns a read-only (constant) reverse iterator that points to the last element in the shape. Iteration is done in reverse element order.
-
inline reverse_iterator rend() noexcept
Returns a read/write reverse iterator that points to one before the first element in the shape. Iteration is done in reverse element order.
-
inline const_reverse_iterator rend() const noexcept
Returns a read-only (constant) reverse iterator that points to one before the first element in the shape. Iteration is done in reverse element order.
-
inline const_iterator cbegin() const noexcept
Returns a read-only (constant) iterator that points to the first element in the shape. Iteration is done in ordinary element order.
-
inline const_iterator cend() const noexcept
Returns a read-only (constant) iterator that points one past the last element in the shape. Iteration is done in ordinary element order.
-
inline const_reverse_iterator crbegin() const noexcept
Returns a read-only (constant) reverse iterator that points to the last element in the shape. Iteration is done in reverse element order.
-
inline const_reverse_iterator crend() const noexcept
Returns a read-only (constant) reverse iterator that points to one before the first element in the shape. Iteration is done in reverse element order.
-
inline void resize(size_t count)
Resizes dimensions container to contain count elements.
-
inline size_t size() const
Returns size of dimension vector. Requires rank to be static.
-
inline iterator insert(iterator position, const Dimension &val)
Returns a read/write iterator that points to the inserted element in the shape.
-
inline void insert(iterator position, size_t n, const Dimension &val)
Inserts count copies of the value before position.
-
template<class InputIterator>
inline void insert(iterator position, InputIterator first, InputIterator last) Inserts elements from range [first, last) before position.
-
inline void reserve(size_t n)
Requests that the dimensions vector capacity be enough to contain n elements.
-
inline void push_back(const Dimension &val)
push element to the end of partial shape
-
template<class ...Args>
inline void emplace_back(Args&&... args) emplace element to the end of partial shape
-
std::string to_string() const
String representation of PartialShape.
Public Static Functions
-
static PartialShape dynamic(Rank r = Rank::dynamic())
Construct a PartialShape with the given rank and all dimensions (if any) dynamic.
- Returns
A PartialShape with the given rank, and all dimensions (if any) dynamic.
-
static bool merge_into(PartialShape &dst, const PartialShape &src)
Try to merge one shape into another.
Merges
src
intodst
, returningtrue
on success andfalse
on failure. Iffalse
is returned, the effect ondst
is unspecified.To merge two partial shapes
s1
ands2
is to find the most permissive partial shapes
that is no more permissive thans1
ors2
, ifs
exists. For example:merge(?,?) -> ? merge(?,{?,?}) -> {?,?} merge({?,?},{?,?}) -> {?,?} merge({1,2,3,4},?) -> {1,2,3,4} merge({1,2},{1,?}) -> {1,2} merge({1,2,?,?},{1,?,3,?}) -> {1,2,3,?} merge({1,2,3},{1,2,3}) -> {1,2,3} merge({1,?},{2,?}) fails [dimension 0 constraints are inconsistent] merge({?,?},{?,?,?}) fails [ranks are inconsistent]
This function (merge_into) performs the “merge” operation described above on
dst
andsrc
, but overwritesdst
with the result and returnstrue
if merging is successful; if merging is unsuccessful, the function returnsfalse
and may make unspecified changes todst
.- Parameters
dst – [inout] The shape that
src
will be merged into.src – The shape that will be merged into
dst
.
- Returns
true
if merging succeeds, elsefalse
.
-
static bool broadcast_merge_into(PartialShape &dst, const PartialShape &src, const ov::op::AutoBroadcastSpec &autob)
Try to merge one shape into another along with implicit broadcasting.
Friends
- friend OPENVINO_API std::ostream & operator<< (std::ostream &str, const PartialShape &shape)
Inserts a human-readable representation of a PartialShape into an output stream.
The output to the stream is in “informal” notation. In other words:
If
shape
has dynamic rank, inserts the string?
.If
shape
has static rank, inserts the string{
, then inserts each dimension ofshape
into the output stream separated by commas, then inserts}
.
PartialShape s1{PartialShape::dynamic())}; PartialShape s2{}; PartialShape s3{1,Dimension::dynamic(),2,3}; PartialShape s4{2,3,4}; std::cout << s1 << std::endl << s2 << std::endl << s3 << std::endl << s4 << std::endl;
? {} {1,?,2,3} {2,3,4}
- Parameters
str – The output stream targeted for insertion.
shape – The shape to be inserted into
str
.
- Returns
A reference to
str
after insertion.
- friend OPENVINO_API PartialShape operator+ (const PartialShape &s1, const PartialShape &s2)
Elementwise addition of two PartialShape objects.
If
s1
ors2
has dynamic rank, returns PartialShape::dynamic().If
s1 and
s2` both have static rank, and their ranks are unequal, throws std::invalid_argument.If
s1
ands2
both have static rank, and their ranks are equal, returns a new shape whosei
th dimension iss1[i] + s2[i]
.
- Parameters
s1 – Left operand for addition.
s2 – Right operand for addition.
- Throws
std::invalid_argument – If
s1
ands2
have inconsistent ranks.- Returns
The result of elementwise adding
s1
tos2
(see description).
-
class PrePostProcessor
- #include <pre_post_process.hpp>
Main class for adding pre- and post- processing steps to existing ov::Model.
This is a helper class for writing easy pre- and post- processing operations on ov::Model object assuming that any preprocess operation takes one input and produces one output.
For advanced preprocessing scenarios, like combining several functions with multiple inputs/outputs into one, client’s code can use transformation passes over ov::Model
Public Functions
Default constructor.
- Parameters
function – Existing function representing loaded model
-
PrePostProcessor(PrePostProcessor&&) noexcept
Default move constructor.
-
PrePostProcessor &operator=(PrePostProcessor&&) noexcept
Default move assignment operator.
-
~PrePostProcessor()
Default destructor.
-
InputInfo &input()
Gets input pre-processing data structure. Should be used only if model/function has only one input Using returned structure application’s code is able to set user’s tensor data (e.g layout), preprocess steps, target model’s data.
- Returns
Reference to model’s input information structure
-
InputInfo &input(const std::string &tensor_name)
Gets input pre-processing data structure for input identified by it’s tensor name.
- Parameters
tensor_name – Tensor name of specific input. Throws if tensor name is not associated with any input in a model
- Returns
Reference to model’s input information structure
-
InputInfo &input(size_t input_index)
Gets input pre-processing data structure for input identified by it’s order in a model.
- Parameters
input_index – Input index of specific input. Throws if input index is out of range for associated function
- Returns
Reference to model’s input information structure
-
OutputInfo &output()
Gets output post-processing data structure. Should be used only if model/function has only one output Using returned structure application’s code is able to set model’s output data, post-process steps, user’s tensor data (e.g layout)
- Returns
Reference to model’s output information structure
-
OutputInfo &output(const std::string &tensor_name)
Gets output post-processing data structure for output identified by it’s tensor name.
- Parameters
tensor_name – Tensor name of specific output. Throws if tensor name is not associated with any input in a model
- Returns
Reference to model’s output information structure
-
OutputInfo &output(size_t output_index)
Gets output post-processing data structure for output identified by it’s order in a model.
- Parameters
output_index – Output index of specific output. Throws if output index is out of range for associated function
- Returns
Reference to model’s output information structure
-
std::shared_ptr<Model> build()
Adds pre/post-processing operations to function passed in constructor.
- Returns
Function with added pre/post-processing operations
-
class Shape : public std::vector<size_t>
- #include <shape.hpp>
Shape for a tensor.
Public Functions
- OPENVINO_API Shape::reference operator[] (std::ptrdiff_t i)
Gets dimension at index.
- Parameters
i – Index to shape dimension [-rank, rank).
- Returns
A reference to i-th dimension of this shape.
- OPENVINO_API Shape::const_reference operator[] (std::ptrdiff_t i) const
Gets dimension at index.
- Parameters
i – Index to shape dimension [-rank, rank).
- Returns
A const reference to i-th dimension of this shape.
- OPENVINO_API Shape::reference at (std::ptrdiff_t i)
Gets dimension at index, with bounds checking.
- Parameters
i – Index to shape dimension [-rank, rank).
- Returns
A reference to i-th dimension of this shape.
- OPENVINO_API Shape::const_reference at (std::ptrdiff_t i) const
Gets dimension at index, with bounds checking.
- Parameters
i – Index to shape dimension [-rank, rank).
- Returns
A const reference to i-th dimension of this shape.
-
struct DiscreteTypeInfo
- #include <type.hpp>
Type information for a type system without inheritance; instances have exactly one type not related to any other type.
Supports three functions, ov::is_type<Type>, ov::as_type<Type>, and ov::as_type_ptr<Type> for type-safe dynamic conversions via static_cast/static_ptr_cast without using C++ RTTI. Type must have a static type_info member and a virtual get_type_info() member that returns a reference to its type_info member.
-
template<typename ForwardIt>