Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MF-1265 - Make Transformer type configurable #1331

Merged
merged 1 commit into from
Jan 20, 2021

Conversation

dborovcanin
Copy link
Collaborator

@dborovcanin dborovcanin commented Jan 16, 2021

Signed-off-by: dusanb94 dusan.borovcanin@mainflux.com

What does this do?

This pull request enables configuring the Transformer type for the Mainflux writer through environment variables.

Which issue(s) does this PR fix/relate to?

Resolves #1265.

List any changes that modify/break current functionality

There are no such breaks.

Have you included tests for your changes?

No.

Did you document any new/modified functionality?

Yes.

@dborovcanin dborovcanin requested a review from a team as a code owner January 16, 2021 17:32
@codecov-io
Copy link

codecov-io commented Jan 16, 2021

Codecov Report

Merging #1331 (ce81b53) into master (0516fe2) will decrease coverage by 0.02%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1331      +/-   ##
==========================================
- Coverage   59.89%   59.87%   -0.03%     
==========================================
  Files         113      113              
  Lines        8835     8835              
==========================================
- Hits         5292     5290       -2     
- Misses       3081     3083       +2     
  Partials      462      462              
Impacted Files Coverage Δ
things/service.go 53.40% <0.00%> (-1.14%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0516fe2...ce81b53. Read the comment docs.

@@ -156,6 +162,21 @@ func newService(session *gocql.Session, logger logger.Logger) consumers.Consumer
return repo
}

func makeTransformer(cfg config, logger logger.Logger) transformers.Transformer {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whould it make sense that this function is actually in Transformer interface?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can't be extracted to the Transformer interface because it would cause circular dependency (implementation imports Transformer to implement it and Transformer imports implementations so that it can instantiate them).

case "SENML":
logger.Info("Using SenML transformer")
return senml.New(cfg.contentType)
case "JSON":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be a constant?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's used only here, so I'd say it's not necessary (it only adds a couple of extra lines of code).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @mteodor
Maybe we can export it from /pkg/messaging? Currently is used only here yes, but also in all writers and probably will be used in other places too. In general, it would be cool to call it messaging.format.senml or similar.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense, but it looks like it's related to this comment. I'm not sure if we should put these constants in pkg/messaging/transformer and have implementation info on the same location where the interface is, or in a separate configuration package. We have a lot of repetitive code in cmd and we should probably fix this in a separate PR attached to issue #1286.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dusanb94 Yea I agree about transformers, but I don't think it's useful only for configuration, either related to transformers.
IMHO it's a simple string, a format type definition that we support (in your PR used for config and related to transformers yes, but not neccearly), It actually belongs to messaging in general (it defines message format type). That's why I see it in /pkg/messaging as exported available format, currently a type string, easily changeable to any other format.

return json.New()
default:
logger.Error(fmt.Sprintf("Can't create transformer: unknown transformer type %s", cfg.transformer))
os.Exit(1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why exit here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, SenML should be also default

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default is probably defined by ENV constant, maybe it is not bad to report error here and give user / deployment engineer info that ENV var has not be set.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, SenML is default and it makes sense to put it here, but the problem is that SenML requires format when instantiation, so defaulting (no env set) would also imply that we set a default format. I'm afraid it's too many implicit default values.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, exit is ok here. Better to exit then trick me to use a format that I didn't want it. It's too critical IMHO

Copy link
Contributor

@drasko drasko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

return json.New()
default:
logger.Error(fmt.Sprintf("Can't create transformer: unknown transformer type %s", cfg.transformer))
os.Exit(1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default is probably defined by ENV constant, maybe it is not bad to report error here and give user / deployment engineer info that ENV var has not be set.

nmarcetic
nmarcetic previously approved these changes Jan 18, 2021
drasko
drasko previously approved these changes Jan 19, 2021
Copy link
Contributor

@drasko drasko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

manuio
manuio previously approved these changes Jan 20, 2021
Copy link
Contributor

@manuio manuio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com>
Copy link
Contributor

@manuio manuio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@manuio manuio merged commit 5cfc930 into absmach:master Jan 20, 2021
@dborovcanin dborovcanin changed the title NOISSUE - Make Transformer type configurable MF-1265 - Make Transformer type configurable Jan 25, 2021
fbugarski pushed a commit to fbugarski/mainflux that referenced this pull request Mar 8, 2021
Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com>
@dborovcanin dborovcanin deleted the configurable-transformer branch June 13, 2022 10:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make Transformers configurable
6 participants