Wyvern makes it so much easier to define and auto retrieve features you need as well as connecting to your model, whether it’s a model that’s served from another service or a model that’s hosted locally within the wyvern pipeline service.
Wyvern has a ModelComponent which is the base component that takes in the inference request and a list of entities, and outputs a model output. The model output is a dictionary mapping entity identifiers to model outputs.
Let’s see how ModelComponent actually works
ModelComponent has an inference function which is the main entrance to model evaluation.
By default, the base ModelComponent slices entities into smaller batches and call batch_inference on each batch. The default batch size is 30. You should be able to configure the MODEL_BATCH_SIZE env variable to change the batch size.
In order to set up model inference, you only need to define a class that inherits ModelComponent and implement batch_inference.
You can also override this function if you want to customize the inference logic.
Here’s an example of defining features you need for your model inference:
The manifest_feature_names
is a @cached_property (you can also just do @property but for cached_property gives a little performance advantage) and is a set of feature strings, with each string in the following format:
realtime_feature_component_name
+ :
+ feature_name
. The realtime_feature_component_name is the name
of the RealtimeFeatureComponent you defined. By default, it is the name of your model class.:
+ FEATURE_NAMEWyvern’s framework will use features defined under manifest_feature_names
As you could see in the example, the RankingModel inherits ModelComponent and defines manifest_feature_names and batch_inference. The batch_inference defines the behavior of model for batch inference. In this example of the _inference_helper
, each inference is pretty much handling the formula we’ve talked about, which is:
This is the basic input format of a ModelComponent.
It contains the request information and the list of entities that’s related to the model
Reference: https://docs.wyvern.ai/sdk_ref#modelinput-objects
After the model is defined, now let’s go back to “Wyvern Pipeline” and see how to set up a pipeline in the code in the next page.
Wyvern makes it so much easier to define and auto retrieve features you need as well as connecting to your model, whether it’s a model that’s served from another service or a model that’s hosted locally within the wyvern pipeline service.
Wyvern has a ModelComponent which is the base component that takes in the inference request and a list of entities, and outputs a model output. The model output is a dictionary mapping entity identifiers to model outputs.
Let’s see how ModelComponent actually works
ModelComponent has an inference function which is the main entrance to model evaluation.
By default, the base ModelComponent slices entities into smaller batches and call batch_inference on each batch. The default batch size is 30. You should be able to configure the MODEL_BATCH_SIZE env variable to change the batch size.
In order to set up model inference, you only need to define a class that inherits ModelComponent and implement batch_inference.
You can also override this function if you want to customize the inference logic.
Here’s an example of defining features you need for your model inference:
The manifest_feature_names
is a @cached_property (you can also just do @property but for cached_property gives a little performance advantage) and is a set of feature strings, with each string in the following format:
realtime_feature_component_name
+ :
+ feature_name
. The realtime_feature_component_name is the name
of the RealtimeFeatureComponent you defined. By default, it is the name of your model class.:
+ FEATURE_NAMEWyvern’s framework will use features defined under manifest_feature_names
As you could see in the example, the RankingModel inherits ModelComponent and defines manifest_feature_names and batch_inference. The batch_inference defines the behavior of model for batch inference. In this example of the _inference_helper
, each inference is pretty much handling the formula we’ve talked about, which is:
This is the basic input format of a ModelComponent.
It contains the request information and the list of entities that’s related to the model
Reference: https://docs.wyvern.ai/sdk_ref#modelinput-objects
After the model is defined, now let’s go back to “Wyvern Pipeline” and see how to set up a pipeline in the code in the next page.