Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IPFS Team Structures - Working & Research Groups (Was: "Proposal: IPFS Working Groups") #285

Merged
merged 13 commits into from
Aug 15, 2018

Conversation

daviddias
Copy link
Member

@daviddias daviddias commented Nov 16, 2017

This is my first stab at designing solid working groups that can tackle different verticals of the IPFS project independently. This comes as a follow up of many informal conversations that many of us had.

The wording still has ways to improve and we should expand on the responsibilities list, specially after checking in with whom has been taking more or less those responsibilities for the last year or so.

There is also an open question on what is the life cycle of a working group and if we are going to have captains for each of these.

UPDATE: See #285 (comment)

@ghost ghost assigned daviddias Nov 16, 2017
@ghost ghost added the in progress label Nov 16, 2017
@ghost
Copy link

ghost commented Nov 28, 2017

Thanks! This looks pretty good to me. Should some of these rather be product teams/groups than working groups? Not all of the groups seem like verticals:

  • in web browsers
  • data structures
  • large data sets & ipfs-cluster (these two should maybe be merged)

The internationalization/localization group could probably be marked as #future.

I guess the next step after this document is for the working groups to constitute, make a repo, and refine+approve their goals?

@mishmosh
Copy link

mishmosh commented Nov 28, 2017

Love the focus on themes. +1 on captains (or max 2 co-captains) for each of these which will help scale.

  • Is this the next evolution of the js-ipfs and go-ipfs divisions? If so, could the proposal also talk about how that transition would work?
  • How will the WG groups communicate & coordinate with each other? Is this the right time to create an IPFS Steering Committee, or can that wait?

@flyingzumwalt
Copy link
Contributor

I would structure this differently. If you don't group these they get confusing, especially when you start listing products like ipfs-cluster alongside ongoing efforts like supporting giant files.

Here's how I think of it, though I'm not bound to the names for each grouping -- we might find better names.

Production & Maintenance

WGs focused on producing and maintaining the technology.

  • Steering
  • Releases
    • Go releases WG
    • JavaScript releases WG
  • QA, Testing and Dev Team Enablement
  • Security
  • Specifications (should be subdivided eventually by spec or category of spec)
  • IPFS Infrastructure
  • Performance and Benchmarking
  • Runtime Support and Build

Community & Usage

WGs focused on supporting adoption and usage of the technology.

  • Community, Events & Evangelism
  • Documentation, Education and Websites
  • Internationalisation and Localization (i18n and l10n)
  • Files (Unixfs), DEX and Data Structures on top of IPLD WG

Ongoing Efforts to Support Specific Uses

  • Integration with Web Browsers WG
  • Large Data Sets WG
  • Porcelain on top of IPFS for dapp developers

Products

Discrete products that have their own release cycles and their own set of stakeholders distinct from IPFS.

  • libp2p
  • IPFS Cluster
  • multiformats
  • IPLD as a project
  • PeerPad, pubsub-room and ipfs-crdt-db ?
  • InterPlanetary TestLab (inactive?)

@flyingzumwalt
Copy link
Contributor

We should also identify which are the key working groups who should be defining at least 1 KR, which ones should define their own set of OKRs, etc.


## Description

The IPFS Project Working Groups are teams of people that are appointed to research, develop and deploy new work under the scope that the working group is focused. This structure is designed to provide clarity and direction to the project, enabling individual contributors to focus their time and energy on the areas they are most interested.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why specifically mention new work? a lot of these WGs do ongoing work that needs perennial attention.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps new was a poor word choice here. I meant new as in their are creating that in the scope. I'll remove it as it is not necessary.

@flyingzumwalt
Copy link
Contributor

Do you mind if I add those headings and generate a Table of Contents to make the doc more readable?


## Current Working Groups

### Release
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Release Working Groups

  • JavaScript Releases
  • Go Releases

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered this and I'm confident that releasing JS and Go can be handled by the same team. In the end, the Release team should focus on auditing the release, ensuring that the quality bar didn't lower (check features added, CI, documentation and so on), that communication with the community and partners happens (changelog, post, twitter, reddit, emails and so on).

Yes, there are some people that have more context of what is happening in Go and others in JS, but in the end it is the same protocol and given that this is a Working Group and not a Working individual, we can have 2~4 people that share enough context to be capable of launching the rocket with the same level of quality.


### Release

The Release Working Group manages the releases for go-ipfs, js-ipfs and its internal dependencies.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should address relationship between IPFS and libp2p, or the document should give an overarching statement about the relationship between IPFS work and libp2p work, and how that impacts the scope of each working group.


### Specifications

The Specifications Working Group coordinates and participates on the specification writting process.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There will also be WGs for each spec or group of specs -- ie. who is on the IPLD Spec WG vs the libp2p spec WG.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the same way that the Internationalization Working Group participants are not the ones writing the translations to every language but rather create the platform and coordinate multiple contributors and possible vendors for those translations to exist, the Specifications working group should be more focused on ensuring that specs are written, clear, and that individuals that participate in other working groups and/or projects can effectively contribute to specs (through training, mentoring and so on).

The Specifications working needs to have more of a role of an Editor + Reviewer, create a Peer Review system and structure the spec writing process (templates, coordination, timelines and objectives).

- Collaborate with with Browser vendors.
- Identify blockers and design constraints of each integration, figuring out new ways to go around those limitations.
- Increase the adoption of the Distributed Web by making it easy for users to access it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Define specifications for address schemes and advocate for those schemes to be adopted

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🌟


- Design Importers and Exporters for different data sets
- Maintain Unixfs
- Improve the APIs that let users interact with Files on IPFS
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • inform and guide users who want to define their own IPLD-based data formats

- Write and make sure that docs are up to date.
- Create Tutorials and Workshops to teach concepts about IPFS and the Distributed Web.
- Develop the front facing Webpages of each project (ipfs.io, libp2p.io, multiformats.io and ipld.io)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • ensure that public forums (discuss.ipfs.io, irc, github, etc) have adequate moderation and responsive support
  • ensure that the code of conduct is followed and provide appropriate support or response when violations occur

@daviddias
Copy link
Member Author

Excellent feedback everyone! Thank you so much for writing these notes, I'm incorporating some and will push in just a bit, but here are some follow-ups to your comments:

Should some of these rather be product teams/groups than working groups? Not all of the groups seem like verticals:

Good point, we might need to define a language to distinguish a team from a working group and have the two classes. I feel here the fundamental question is that there will be endeavors that require different cycles of feedback and lifetime. We can either use different words to distinguish those or we can use always the term Working Group and then tag them with different properties through labels.

The internationalization/localization group could probably be marked as #future.

Yes, I agree that it is better for us to get really well defined working groups that currently have already people working on them rather than trying to boot all of them at the same time and put a lot of responsibility in few individuals.

I guess the next step after this document is for the working groups to constitute, make a repo, and refine+approve their goals?

Once we are happy with this document and get it merged, we can move to that step, yes :)

Love the focus on themes. +1 on captains (or max 2 co-captains) for each of these which will help scale.

Thank you, I feel that it is a fundamental little detail that can improve a ton how we grow and rotate the folks working on this groups.

Is this the next evolution of the js-ipfs and go-ipfs divisions? If so, could the proposal also talk about how that transition would work?

Yes, for js-ipfs, go-ipfs and the whole project as a whole. Currently, there is a lot of work that falls on the go and js IPFS leads that can be lead by working groups. Once these are formed, js-ipfs and go-ipfs will no longer need to be a team, but rather a collection from different individuals that contribute to the project, while the Releases working group ensures that quality gets merged and released.

How will the WG groups communicate & coordinate with each other? Is this the right time to create an IPFS Steering Committee, or can that wait?

I've received some feedback with uncertainty about the value of a Steering Committee in the long term. I still strongly believe that having such Committee will valuable for the long-term vision and facilitation between working groups, however, we can totally live without it for a bit more time and at the same time start formalizing the working groups around it.

I would structure this differently. If you don't group these they get confusing, especially when you start listing products like ipfs-cluster alongside ongoing efforts like supporting giant files.
Here's how I think of it, though I'm not bound to the names for each grouping

Grouping sounds good to me, especially because this list will tend to increase overtime.

Nevertheless, I would like everyone to consider a slow start for creating working groups. Let's focus on the ones that are already becoming a thing (Security, Releases, ipfs-cluster, IPFS in Web Browsers) and work with the folks on those working groups to polish our OKR setting process and then create the template for more working groups to start.

We should avoid having one person being responsible for many (many === number that exceeds the capacity of an individual to actively manage and set goals for that working group).

@daviddias
Copy link
Member Author

Updated the doc with the suggestions made :)

@ghost ghost assigned flyingzumwalt Nov 29, 2017

#### Release

The Release Working Group manages the releases for go-ipfs, js-ipfs and its internal dependencies.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want this to be one working group or do you want it to be separate release working groups for each language?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I provided an answer on your previous comment, perhaps it got hidden with the rearranging. Here you go #285 (comment)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered this and I'm confident that releasing JS and Go can be handled by the same team. In the end, the Release team should focus on auditing the release, ensuring that the quality bar didn't lower (check features added, CI, documentation and so on), that communication with the community and partners happens (changelog, post, twitter, reddit, emails and so on).

Yes, there are some people that have more context of what is happening in Go and others in JS, but in the end it is the same protocol and given that this is a Working Group and not a Working individual, we can have 2~4 people that share enough context to be capable of launching the rocket with the same level of quality.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding to the above, there should be some work on a release that involves checking spec compliance and interoperability.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defining the release roadmap and deciding what actually goes into a release feels like a responsibility of the dev team for that implementation, since they ultimately know when something is ready to be released or not.

With the rest of the points, I feel like we can fold that into QA, as it got to do with some sort of automated pipelines, testing and feedback cycles.

- Test releases.
- Announce and gather feedback from the community on each release.

#### Specifications
Copy link
Contributor

@flyingzumwalt flyingzumwalt Nov 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having a generic "Specifications" WG invites the same kind of confusion we have with a generic "javascript team". People work on the specs that are relevant to their work and don't necessarily need to be involved in other specs. Do you intend to have a small "specifications" WG that deals with all specs or is this a large group that includes everyone who has these kinds of responsibilities for any spec?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you get to see my response to your last comment on the Specifications Working Group? Here #285 (comment)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reposting it here to make it easier to find:

In the same way that the Internationalization Working Group participants are not the ones writing the translations to every language but rather create the platform and coordinate multiple contributors and possible vendors for those translations to exist, the Specifications working group should be more focused on ensuring that specs are written, clear, and that individuals that participate in other working groups and/or projects can effectively contribute to specs (through training, mentoring and so on).

The Specifications working needs to have more of a role of an Editor + Reviewer, create a Peer Review system and structure the spec writing process (templates, coordination, timelines and objectives).

@flyingzumwalt
Copy link
Contributor

Thing to add to each of these: What are this WG's responsibilities for contributing to the quaterly & annual planning process? What are their contributions to setting each quarter's OKRs?

@daviddias
Copy link
Member Author

Shall we schedule a call to identify what is left for us to do so that we can move confidently with Working Group structured planning for next quarter?

- Create and monitor continuous testing for the multiple supported platforms (Architecture, OS'es, Browsrs).
- Explore new runtimes and create Roadmaps towards supporting those.

#### QA, Testing and Dev Team Enablement
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@victorbjelkholm did you had the chance to review this PR? Let's sync up on what this means for 2018 quarter OKRs on QA, Testing and Dev Team Enablement :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@diasdavid will review today.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All seems good to me 👍

@jbenet
Copy link
Member

jbenet commented Jan 4, 2018

thanks very much @flyingzumwalt

Likewise for IPLD, though we were already forming the Data Structures + IPLD WG, so we can probably just proceed as planned with that WG defining its OKRs but maybe we want to be clear that its scope extends beyond IPFS needs...

The IPLD work i have in mind is different in nature than what's listed here -- IPLD can include those pieces, but those pieces (files/dex) are much more on IPFS camp than IPLD. (definitely an interface between the two and could fall in either).

The "direct IPLD work" is about:

  • IPLD Data Model
  • IPLD Type System
  • IPLD Selectors
  • IPLD Transformations
  • IPLD interfaces
  • IPLD implementations (in many languages)
  • distributed computing languages
  • verifiable computation on top of IPLD
  • proof-carrying code

more notes

  • DEX and Files are applications of IPLD, and definitely can fall into the IPLD group (as they can fall into the IPFS group). But they're not the defining things.

I propose:

  • Forming an "IPLD" WG explicitly (what i mentioned above)
  • Forming a "IPFS Internal Data Structures" WG (unixfs, DEX, and so on)
    • (though if too much overhead, just lump this into one for now, but be explicit the group is IPLD (much more general))

cc @Stebalien @nicola

@daviddias daviddias added P1 High: Likely tackled by core team if no one steps up and removed in progress labels Jun 2, 2018
@ghost ghost assigned daviddias Aug 12, 2018
@ghost ghost added the in progress label Aug 12, 2018
@daviddias daviddias changed the title Proposal: IPFS Working Groups IPFS Team Structures - Working & Research Groups (Was: "Proposal: IPFS Working Groups") Aug 12, 2018
@daviddias
Copy link
Member Author

daviddias commented Aug 12, 2018

A lot has happened since these proposal was originally posted (November 2017).

I've updated the doc to change the character of it from proposal to actually just capture what exists today and how we have been operating, creating a foundation for everyone that is joining the project now.

> This Working Group hasn't been formed yet.

- **Coordination**: https://github.com/ipfs/community
- **Captain**: ???
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure if I would leave this one around or not. The truth is that there is a lot of work done and planned, but we don't have an official Captain for this front.

@mikeal given that you are the person leading most of the new charges, would you like to take this role and plan OKRs for next quarter? Knowing that this is a role that you are also actively looking to fill/replace yourself?

> This Working Group hasn't been formed yet.

- **Coordination**: https://github.com/ipfs/docs
- **Captain**: ???
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was also unsure if I would leave this one around. Again, truth is that there has been work and even structure, but since Q3, we no longer have OKRs for Documentation. @Mr0grog was there a change of direction? Is the Docs front finished or going to be pursued in a completely different way?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugh, sorry for missing this and not replying.

was there a change of direction? Is the Docs front finished or going to be pursued in a completely different way?

I don’t know. The specific projects I was contracted to do are done and I’ll be gone in a week (after, obviously, an absurdly late delivery). I did not do Q3 OKRs because I’m not here for all of Q3 and nobody asked me to, so it seemed like it would be setting up expectations I could not fulfill. I think/hope the thread of this work is going to be picked up by @meiqimichelle and/or @mikeal, but I really can’t speak for future directions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi all! I just finished writing up my thoughts regarding the docs work here: https://github.com/ipfs/user-research/tree/master/distributed-kit . Long story short, setting up and staffing a structure to make sure @Mr0grog 's work continues would address our biggest usability problem, and our biggest blocker to user success, in the IPFS ecosystem. @mikeal and IPFS leadership (not sure exactly who, for this convo) should connect and align on how to staff next steps on this work.

I know @diasdavid and @flyingzumwalt are short on time right now. Perhaps I'll reach out to @mikeal first to get his thoughts, and then broaden the convo with a proposal rather than waiting for a time when all are available.

@daviddias daviddias force-pushed the working-groups branch 3 times, most recently from 23cd7ec to 2659183 Compare August 12, 2018 15:02

Each Working Group is free to experiment with setting their own pace, tracking work and defining priorities. The only requirements are that the Working Group exposes its focus through OKRs to the rest of the org (common interface), that it assigns a Captain, creates an entry point repo and has at least 2 full time contributors dedicatted to it.

Each contributor shouldn't carry responsibilities accross multiple working groups. This is not forbidden by any means but it is greatly disencouraged as it will disable the contributor from achieving full focus.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit confusing as what seems to be outlined as Working Groups here, is what I believed "Teams" was before.

One of the OKRs for the Testing WG this quarter is "Have at least one member from each team be a part of the Working Group" which is based on the fact that I thought Working Groups are supposed to have members who can also be a part of other working groups. I think that's the "common" definition of a working group as well, as set of people who can participate in many working groups.

It's also unclear where the "IPFS Core" Working Group would fit into this here, since people activate in that, needs to be active in other working groups as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit confusing as what seems to be outlined as Working Groups here, is what I believed "Teams" was before.

There hasn't been a Teams definition before.

One of the OKRs for the Testing WG this quarter is "Have at least one member from each team be a part of the Working Group"

In reality, it is still OKRs of the Dev Team Working Group and not complete different contexts

> This Working Group hasn't been formed yet.

- **Coordination**: https://github.com/ipfs/community
- **Captain**: ???
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put me down as the Captain here. Is there a doc of best practices for getting a WG up and running?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


**Responsibilities include**:

- Organize Meetups in multiple cities.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove this? The next point on helping community members organize meetups in their own communities covers this.

Alternatively, we could alter this to focus on larger events. "Support IPFS related conferences."

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@daviddias
Copy link
Member Author

I'm going to move ahead and merge this doc as is. As described in #285 (comment), this document changed from proposal to just capture what got developed almost organically over the last 9 months.

@daviddias daviddias merged commit 7f7536a into master Aug 15, 2018
@daviddias daviddias deleted the working-groups branch August 15, 2018 16:54
@ghost ghost removed the in progress label Aug 15, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 High: Likely tackled by core team if no one steps up
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants