feat: Add transformations server endpoint to feature_store.yaml#85
feat: Add transformations server endpoint to feature_store.yaml#85wparsley-eg wants to merge 10 commits intomasterfrom
Conversation
| go_feature_retrieval: Optional[bool] = False | ||
| """ If True, use the embedded Go code to retrieve features instead of the Python SDK. """ | ||
|
|
||
| go_transformations_server: Optional[bool] = True |
There was a problem hiding this comment.
Had a look at the repo_config. Noticed feature_server section in repo_config. We should have grouped these configs there instead of separate configs.
There was a problem hiding this comment.
We are adding expedia provider. With that we can always force the flag to TRUE irrespective of what is set in feature_store.yaml. So we can ignore the above comment "Earlier logic was handling: Flag=False and Endpoint=Value, Its using Transformation service. Now it's using GOPY bindings"
There was a problem hiding this comment.
What about moving configurations to feature_server section?
| endpoint := fmt.Sprintf("%s-transformations.%s.svc.cluster.local:80", sanitizedProjectName, productName) | ||
| transformationService, _ := transformation.NewGrpcTransformationService(config, endpoint) | ||
|
|
||
| var transformationService *transformation.GrpcTransformationService = nil |
There was a problem hiding this comment.
Earlier logic was handling: Flag=False and Endpoint=Value, Its using Transformation service. Now it's using GOPY bindings. Was that the intention when the code was written previously?
There was a problem hiding this comment.
No, this version of the code is what should be done. If flag=false, we should be using GOPY bindings. Flag should control whether we use the transformations server or not.
|
Closing in favor of this pr |
3 participants
What this PR does / why we need it:
Currently, the transformations server endpoint is only configurable through environment variables. This will allow users to fully specify the endpoint to be used.
Why put this in feature_store.yaml?
I studied the Java feature server, which includes its own transformations server. The Java feature server is a Springboot application, and therefore uses its Helm charts to specify the endpoint and port for the transformations server.
However, there is no Springboot equivalent in Go. Go developers tend to prefer a more spartan style of development and ingesting variables from the Helm charts would have to be done manually, and may not be portable depending on use cases.
However, the Go process already ingests the feature_store.yaml file, and we can easily set the value there. This requires no extra code and should be more portable for open source users.