Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MF-397 - Introduce Thing Groups #1259

Merged
merged 50 commits into from
Nov 23, 2020
Merged

MF-397 - Introduce Thing Groups #1259

merged 50 commits into from
Nov 23, 2020

Conversation

mteodor
Copy link
Contributor

@mteodor mteodor commented Oct 16, 2020

Signed-off-by: Mirko Teodorovic mirko.teodorovic@gmail.com

Adding group of things

@mteodor mteodor marked this pull request as ready for review October 21, 2020 08:33
@mteodor mteodor requested a review from a team as a code owner October 21, 2020 08:33
things/api/logging.go Outdated Show resolved Hide resolved
things/api/things/http/transport.go Show resolved Hide resolved
things/api/things/http/transport.go Show resolved Hide resolved
things/api/things/http/transport.go Show resolved Hide resolved
things/api/metrics.go Outdated Show resolved Hide resolved
things/groups.go Outdated
Description() string
Metadata() Metadata

SetID(id string)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need all these setters? Why not passing group struct pointer and changing the value directly.

This looks like some OO encapsulation, maybe it is needed but it Go it gives me "code smell".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idea here and in things/http.go is to extract the common code that can be used for other entities that we wish to group
so Group is an interface that would have its implementations in each entity package itself
Of course if this solution is ok, http.go and groups.go would be moved to appropriate package

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is OK, but I do not like these setters. You can create an var struct without the setters, and then change internal struct directly through pointer receiver.

things/api/logging.go Outdated Show resolved Hide resolved
@dborovcanin dborovcanin changed the title NOISSUE - Add things group MF-397 - Introduce thing groups Oct 22, 2020
things/api/logging.go Outdated Show resolved Hide resolved
things/api/metrics.go Outdated Show resolved Hide resolved
things/groups.go Outdated Show resolved Hide resolved
things/groups.go Outdated
"context"
)

type Group interface {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also not sure about this idea. Why can't we use Group struct here? This interface resembles struct anyways (explains fields, rather than behavior). It is a limitation compared to the interface, but it also provides better type safety and this looks sufficient for Mainflux needs anyways.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because Thing groups and User groups could differ in implementation and this interface should be common denominator.
Idea was to have at least some parts modular, that I put this into a package and reuse it in Users, Things and Channels groups, no need to rewrite again each endpoint for example.
This is something that was asked for by @drasko, at least one part. I can easily copy paste and refactor this into the usual place

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mteodor this is OK, but remove getters and setters, use strucs and change the fields directly via pointer receiver.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Group entity itself, however, is the same in both cases, right?

things/postgres/groups.go Outdated Show resolved Hide resolved
@mteodor mteodor changed the title MF-397 - Introduce thing groups MF-397 - Introduce Thing Groups Oct 23, 2020
things/service.go Outdated Show resolved Hide resolved
things/tracing/groups.go Outdated Show resolved Hide resolved
internal/groups/errors.go Outdated Show resolved Hide resolved
internal/groups/errors.go Outdated Show resolved Hide resolved
internal/groups/groups.go Outdated Show resolved Hide resolved
internal/groups/groups.go Outdated Show resolved Hide resolved
things/api/logging.go Outdated Show resolved Hide resolved
things/api/logging.go Outdated Show resolved Hide resolved
things/postgres/things.go Outdated Show resolved Hide resolved
things/service.go Outdated Show resolved Hide resolved
Comment on lines 52 to 56
// ListChildren retrieves groups that are children to group identified by parentID
ListChildren(ctx context.Context, token, parentID string, offset, limit uint64, m Metadata) (GroupPage, error)

// ListParents retrieves groups that are parent to group identified by childID.
ListParents(ctx context.Context, token, childID string, offset, limit uint64, m Metadata) (GroupPage, error)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I understand that using parents and children terminology is natural, I'm afraid it's also ambiguous because parents imply that one Group has multiple parents, and children implies only direct descendants.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, did you consider adding a feature to search genealogy only up to some depth, something like: give me only my children? Or: give me my only my children and children of my children. Same for parents. There are many possibilities for searching group members, but let's not get into that in this PR, this is just a remark for future reference.

Copy link
Collaborator

@dborovcanin dborovcanin Oct 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding https://github.com/mainflux/mainflux/pull/1259/files#r511910630, do Service methods return direct ascendents and descendants or the entire tree? This should be better explained in the documentation comments so that the one who implements knows the desirable behavior of the Service interface.

internal/groups/http.go Outdated Show resolved Hide resolved
internal/groups/http.go Outdated Show resolved Hide resolved
internal/groups/http.go Outdated Show resolved Hide resolved
internal/groups/http.go Outdated Show resolved Hide resolved
pkg/sdk/go/things_test.go Show resolved Hide resolved
internal/groups/groups.go Show resolved Hide resolved
things/api/logging.go Outdated Show resolved Hide resolved
things/postgres/init.go Outdated Show resolved Hide resolved
things/service.go Outdated Show resolved Hide resolved
internal/groups/api/endpoint.go Outdated Show resolved Hide resolved
internal/groups/api/endpoint.go Outdated Show resolved Hide resolved
bootstrap/mocks/things.go Show resolved Hide resolved
internal/groups/api/responses.go Outdated Show resolved Hide resolved
internal/groups/api/endpoint.go Outdated Show resolved Hide resolved
internal/groups/errors.go Show resolved Hide resolved
pkg/sdk/go/things_test.go Show resolved Hide resolved
things/api/things/http/transport.go Show resolved Hide resolved
things/postgres/channels.go Show resolved Hide resolved
Comment on lines 52 to 56
// ListChildren retrieves groups that are children to group identified by parentID
ListChildren(ctx context.Context, token, parentID string, offset, limit uint64, m Metadata) (GroupPage, error)

// ListParents retrieves groups that are parent to group identified by childID.
ListParents(ctx context.Context, token, childID string, offset, limit uint64, m Metadata) (GroupPage, error)
Copy link
Collaborator

@dborovcanin dborovcanin Oct 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding https://github.com/mainflux/mainflux/pull/1259/files#r511910630, do Service methods return direct ascendents and descendants or the entire tree? This should be better explained in the documentation comments so that the one who implements knows the desirable behavior of the Service interface.

things/postgres/groups_test.go Outdated Show resolved Hide resolved
things/postgres/groups_test.go Outdated Show resolved Hide resolved
things/postgres/groups.go Show resolved Hide resolved
things/postgres/groups.go Outdated Show resolved Hide resolved
internal/groups/api/requests.go Outdated Show resolved Hide resolved
internal/groups/api/requests.go Outdated Show resolved Hide resolved
internal/groups/api/requests.go Outdated Show resolved Hide resolved
things/postgres/groups.go Outdated Show resolved Hide resolved
internal/groups/groups.go Outdated Show resolved Hide resolved
internal/groups/groups.go Outdated Show resolved Hide resolved
internal/groups/groups.go Outdated Show resolved Hide resolved
internal/groups/groups.go Outdated Show resolved Hide resolved
internal/groups/api/endpoint.go Outdated Show resolved Hide resolved
things/service.go Outdated Show resolved Hide resolved
things/tracing/groups.go Outdated Show resolved Hide resolved
things/tracing/groups.go Outdated Show resolved Hide resolved
things/tracing/groups.go Outdated Show resolved Hide resolved
things/tracing/groups.go Outdated Show resolved Hide resolved
internal/groups/api/requests.go Outdated Show resolved Hide resolved
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
return nil, errors.Wrap(groups.ErrFailedDecode, err)
}
req.ID = bone.GetValue(r, "groupID")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here it is

users/service.go Outdated Show resolved Hide resolved
things/tracing/groups.go Outdated Show resolved Hide resolved
things/tracing/groups.go Outdated Show resolved Hide resolved
things/service.go Outdated Show resolved Hide resolved
things/postgres/things.go Outdated Show resolved Hide resolved
things/postgres/init.go Outdated Show resolved Hide resolved
things/postgres/groups_test.go Outdated Show resolved Hide resolved
things/postgres/groups_test.go Outdated Show resolved Hide resolved
things/postgres/groups_test.go Outdated Show resolved Hide resolved
things/postgres/groups_test.go Outdated Show resolved Hide resolved
@codecov-io
Copy link

codecov-io commented Nov 6, 2020

Codecov Report

Merging #1259 (e8b6a8d) into master (663bd4e) will decrease coverage by 3.14%.
The diff coverage is 19.53%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1259      +/-   ##
==========================================
- Coverage   67.52%   64.38%   -3.15%     
==========================================
  Files         108      109       +1     
  Lines        7316     7819     +503     
==========================================
+ Hits         4940     5034      +94     
- Misses       1936     2346     +410     
+ Partials      440      439       -1     
Impacted Files Coverage Δ
things/postgres/groups.go 0.00% <0.00%> (ø)
things/redis/streams.go 85.59% <0.00%> (-8.80%) ⬇️
things/service.go 54.28% <1.85%> (-23.41%) ⬇️
things/api/things/http/transport.go 90.02% <98.50%> (+2.30%) ⬆️
things/postgres/channels.go 79.76% <100.00%> (+0.08%) ⬆️
things/postgres/init.go 95.23% <100.00%> (+2.25%) ⬆️
things/postgres/things.go 79.42% <100.00%> (ø)
users/service.go 66.42% <100.00%> (+0.70%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 663bd4e...e8b6a8d. Read the comment docs.

return groupRes{}, errors.Wrap(groups.ErrMalformedEntity, err)
}

group := groups.Group{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we use group and in the function before g. Please choose one.

internal/groups/api/endpoint.go Outdated Show resolved Hide resolved
internal/groups/api/endpoint.go Outdated Show resolved Hide resolved
internal/groups/api/endpoint.go Outdated Show resolved Hide resolved
internal/groups/api/endpoint.go Outdated Show resolved Hide resolved
func DecodeGroupCreate(_ context.Context, r *http.Request) (interface{}, error) {
if !strings.Contains(r.Header.Get("Content-Type"), contentType) {
return nil, groups.ErrUnsupportedContentType
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add empty line below.

var req createGroupReq
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
return nil, errors.Wrap(groups.ErrFailedDecode, err)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add empty line below.

if !strings.Contains(r.Header.Get("Content-Type"), contentType) {
return nil, groups.ErrUnsupportedContentType
}
var req updateGroupReq
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

things/service.go Outdated Show resolved Hide resolved
things/service.go Outdated Show resolved Hide resolved
internal/groups/api/endpoint.go Outdated Show resolved Hide resolved
if err := req.validate(); err != nil {
return groupPageRes{}, errors.Wrap(groups.ErrMalformedEntity, err)
}
page, err := svc.ListParents(ctx, req.token, req.groupID, req.level, req.metadata)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please stay consistent in adding empty lines comparing to other function.

internal/groups/api/endpoint.go Outdated Show resolved Hide resolved
child := toViewGroupRes(*ch)
view.Children = append(view.Children, &child)
}
return view
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If empty line was added above, add one here as well

if req.token == "" {
return groups.ErrUnauthorizedAccess
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If function below do not have white lines, please suppers these as well.

Copy link
Contributor

@manuio manuio Nov 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be consistent I'd add it in the function where it's missing.

drasko
drasko previously approved these changes Nov 19, 2020
Copy link
Contributor

@drasko drasko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

manuio
manuio previously approved these changes Nov 19, 2020
Copy link
Contributor

@manuio manuio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>
Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>
@mteodor mteodor dismissed stale reviews from manuio and drasko via a3b52c5 November 19, 2020 12:03
Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>
manuio
manuio previously approved these changes Nov 19, 2020
Copy link
Contributor

@manuio manuio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PRIMARY KEY (owner_id, path),
FOREIGN KEY (parent_id) REFERENCES thing_groups (id) ON DELETE CASCADE ON UPDATE CASCADE
)`,
`CREATE TABLE IF NOT EXISTS id (
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Table name id is less than ideal, consider renaming to members.

q := fmt.Sprintf(`SELECT g.id, g.name, g.owner_id, g.parent_id, g.description, g.metadata, g.path, nlevel(g.path) as level FROM
(SELECT thing_groups.path FROM thing_groups where id = :parent_id) parent LEFT JOIN
(SELECT id, name, owner_id, parent_id, description,metadata, path FROM thing_groups) g
ON g.path @> parent.path AND nlevel(parent.path) - nlevel(g.path) >= 0 AND nlevel(parent.path) - nlevel(g.path) <= :level %s`, mq)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is g.path @> parent.path AND nlevel(parent.path) - nlevel(g.path) >= 0 necessary, or we can use only g.path @> parent.path?

mq = fmt.Sprintf("AND %s", mq)
}

q := fmt.Sprintf(`SELECT g.id, g.name, g.owner_id, g.parent_id, g.description, g.metadata, g.path, nlevel(g.path) as level FROM
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using simpler self join. You can try this query:

SELECT g.id, g.name, g.owner_id, g.parent_id, g.description, g.metadata, g.path, nlevel(g.path) as level
FROM thing_groups parent, thing_groups g
WHERE parent.id = :id AND g.path @> parent.path AND nlevel(parent.path) - nlevel(g.path) <= :level;

It's a little simpler (and shorter) and possibly more performant.

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>
manuio
manuio previously approved these changes Nov 20, 2020
Copy link
Contributor

@manuio manuio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>
Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>
blokovi
blokovi previously approved these changes Nov 20, 2020

type groupRes struct {
id string
created bool
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing updated?


type groupRes struct {
id string
created bool
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

created is a time, not bool. Time of creation.

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>
Path string
Children []*Group
CreatedAt time.Time
UpdateAt time.Time
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UpdatedAt

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>
dborovcanin
dborovcanin previously approved these changes Nov 23, 2020
Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>
Copy link
Contributor

@drasko drasko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@drasko drasko merged commit 7195cad into absmach:master Nov 23, 2020
@mteodor mteodor mentioned this pull request Dec 1, 2020
@mteodor mteodor deleted the things-group branch September 13, 2021 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants