Tide development plan

[yoshuawuyts]

As a follow-up to #325, I'd like to share the plan of how we can get Tide to a publishable state. This is not so much an issue to ask for input, but more to share what the plan is in case people are confused by what's going on.

The core problem I think we currently have is one of my design: we have too many different crates, which makes it hard to publish & manage. In contrast: async-std has all functionality as part of a single repo, and adds new features behind the "unstable" feature flag. I think we should consolidate Tide into a single crate, and introduce an "unstable" feature flag.

I think we should very much do the same, and the easiest way to get there is by resetting the repo by a few dozen commits to 4a53890, the commit right before all the "split" PRs landed.

Note that this was fantastic work, implemented by @prasannavl. But I think this wasn't the right direction, which is entirely on me.

From there we can move back through the commit log and start applying patches that have landed before. I estimate this to be a few uninterrupted days worth of work.

---

From there the plan is less clear. But I have some idea of API changes I'd like to make. But before we do that we should probably create a new "snapshot" release so people can work off the latest master branch.

I'd like to spend some effort during all this to write docs. We don't really have much right now, and I think that's holding us back. Unsure about the exact timeline, but it's something I think needs to be prioritized.

This all sounds cool, how can I help?

Unfortunately I think there's not much that can be done right now. The only separate issue that's coming up is http-rs/http-service#31; so if you want to pick this up and help port http-service that would be fantastic. Please note though that I won't have any bandwidth to mentor this, and it won't be easy.

Other than that this is mostly me effectively taking a lock on the project to move things forward. Once that's over I should have a better idea of which steps to take after, and tracking issues can be written. That's all further out still though. More updates in the future!

Conclusion

This is the current work plan I'm thinking of executing on. This means some big changes, but the path seems clear.

[yoshuawuyts]

A permanent archived branch of the current HEAD of Tide can be found here: https://github.com/http-rs/tide/tree/archive-split

[yoshuawuyts]

Now that 0.3.0 has been released, the work begins to rewind prior patches onto the current master in order to release 0.4.0. In order we should do the following patches.

Sourced from the PR page:

Page 3

• Fix example link in readme #221 Fix example link in readme.
• add tide-log #222 Add Tide log.
• Remove #[allow(unused_mut)] #231 Remove #[allow(unused_mut)]
• Improve curl command consistency #233 Improve curl command consistency
• Rename trait ExtractForms to ContextExt #239 Rename trait ExtractForms to ContextExt
• Adjust code quality enforcement #241 Adjust code quality encorcement
• Fix broken links in readme #242 Fix broken links in readme
• remove async box macro #245 remove async box macro
• Sweep the cache before uploading it #246 Sweep the cache before uploading it
• Add doc comment #250 Add doc comment
• pub use ResultDynErrExt in tide::error #253 pub use ResultDynErrExt in tide::error

Page 2

• Fix the documentation for wildcards for path definitions to match implementation #254 Fix the documentation for wildcards for path definitions
• Fix links in README #256 Fix links in readme
• Rename Context<Data> to Context<State> #260 rename Context to Context
• Middleware-based compression and decompression #194 Compression middleware
• Propagate hyper error on serve failure #261 Propagate Hyper error on serve failure
• Implemented CORS header handler middleware #262 Implemented CORS header handler middleware
• Add middleware for catching panics #265 Add middleware for catching panics
• Add templating example with tera #267 Add templating example with tera
• Adjust docs for consistency #272 Adjust docs for consistency
• Cors middleware extension #275 Cors middleware extension
• Tide on runtime #283 Tide on Runtime
• Set StatusCode for an empty response from () to 204 #285 Set statuscode for an empty response from () to 204
• Rewrite tide-slog to setup a per-request logger instance and support slog-scope #287 Update tide-slog
• Add doc example showing query parameter extraction #291 Add doc example showing query param extraction

Page 1

• querystring: replace dependency serde_urlencoded with serde_qs #299 Replace serde_urlencoded with serde_qs
• removed existential_type feature #301 Remove existential_type feature
• Fix clippy error in static file example #303 Fix clippy error in static file exmaple
• cookies: is &mut self required? #317 Cookies: is &mut required?
• CORS middleware: better handling of Origin #318 CORS Middleware: better handling of Origin
• Nested router example #331 Nested router example
• feat(core): If JSON Serialize failure return 500 and Error message #332 if JSON serialize failure return 500 and error message
• Remove dev dependency on the runtime crate #334 Remove dev dependency on Runtime crate

[prasannavl]

I'm very cautious of any projects that do hard resets. (Infact, would go so much to say that I'd never ever do that in public projects with other contributors unless there's a serious security issue).

This simply dismissed every contributor's work from the point of the reset (they will no longer be accredited for their work). Note that I do not have problems with the decision to rewind as such, even though I wish this was experimented/discussed more transparently and for slightly longer time - however, speaking for myself, I was busy and barely involved with WG for over 2 months, and as such I have no problems with decisions being made without being expected to be kept in the loop, but I think the way in which they were made and executed here raises questions.

If it had to be done, this IMO should have been done with reverts that don't make the contributions of others disappear into thin air. Instead, what happened here, I think goes against the nature of good open source software conduct. Erasing the work of others who had invested time and code into the project due to it's nature of association to the WG, just because it no longer is, makes me very uneasy.

[yoshuawuyts]

@prasannavl I hear you, and don't disagree. However I don't quite have the ability to use git in such a way that I know how to replay commits done on separate files on top of a different file tree. It might be possible, but it'd definitely take a while.

I feel the biggest loss of work is probably yours though, as you're the person that executed the project split. I'm deeply appreciative for the work you've done, and think you did it well. However the design left us in a state where it was hard to publish, and as such it became apparent after the fact that we had to change course again.

What I've done here is to try and get us to a publishable state, within the best of my ability. I wanted this to be in a timescale of days, not weeks or months as I estimated approaches would take. I understand this may not be entirely satisfactory, but I hope you can empathize with the desire for Tide to remain an active project and keep moving ahead.

[dtolnay]

In case this is only about lack of git foo, I pushed https://github.com/dtolnay/tide/commit/1b67aa43879a728854d31dfae3a451cfae31abb5 which has contents that are identical to current master but doesn't discard any of the project history.

[prasannavl]

@dtolnay thank you for providing the neat solution! that if rebased upon, pretty much addresses my concern.

@yoshuawuyts - personally, I'd feel much more comfortable rebasing on top of that above whenever you get the time. I wrote my previous comment at the risk of being excessively nitpicky there, but for a healthy open collaboration, I quite strongly believe erasing history should be avoided as a matter of principle.

On the changes itself and my work on the split being the most to go - honestly, I couldn't care less about it - things being changed and the discovery of the sweet spot is a part of the lifecycle of pre v1.0 projects and have no objections to any of it whatsoever. While I evidently had a different direction in my mind, I also wasn't able to put the time into it to follow them up with larger contributions.

I personally see single crate, or multiple crates to be minor issues in the grand scheme of things, compared to practical functionality and progress of the project especially when it has stagnated. It's a lot of work either way, and if you think all this reversal will help bring more progress, I'm just glad that you've decided to take this head on and move this forward in whichever way you see as the best fit accordingly. And thank you for putting in your time to get it moving!

[yoshuawuyts]

All commits have been rewound. Waiting on CI to pass before merging the third batch, and then we should be good.

Going to pick this back up next week. Probably first step is to do a general pass and do some spring cleaning. The current state of docs isn't great, and some reorganization would go a long way. Once that's done we should probably do a snapshot release again.

The step after that is to do a step-by-step review of each API, and mark everything outside of Tide core as "unstable". We then have some freedom to edit APIs, and slowly stabilize them.

Somewhere along this process we should also address http-rs/http-service#31, but I'm not sure when the right time is. Either way, the first few steps seem clear.

Thanks for your patience y'all!

[rrichardson]

Thanks for your hard work on this. I really appreciate your transparency and I like the choices you've made regarding the developer/user centric focus on the API.

Is this the correct place to discuss an HTTP2 strategy? Apologies if it is not.

Adoption of HTTP2 is quite pervasive now, and there are performance and usability advantages to be had. Stats show that almost 50% of websites support it, and I think all browsers and most CLI clients support it.

I am not demanding that we incorporate it immediately, but it would be good to get a high level architectural picture about how I, as a developer, could hope to have a single set of tools to manage http/2 upgrade/fallback and ALPN, and be able to have my routes/handlers serve requests without regard for the underlying protocol.

[yoshuawuyts]

@rrichardson o/ thanks for raising! We definitely want to support HTTP/2, but I'd like us to be deliberate about it and ensure the foundation is in place for us to add it. This includes TLS, WebSockets, and error handling first.

If we were to add it I'd probably want to us to have the ability to use nghttp2 for this. I started initial work on creating high-level bindings to it in https://github.com/http-rs/nghttp2; but that work needs to be finished. If you're interested in progressing this, help would be very welcome!

[kellytk]

https://github.com/http-rs/nghttp2

@yoshuawuyts Why put effort into bindings for C-based code rather than using Rust exclusively?

[yoshuawuyts]

@kellytk nghttp2 is the most widely deployed HTTP/2 implementation used by Curl, Node.js, and Nginx alike. Being able to rely on the same impl would be quite nice.

Though perhaps you're right; the argument for having nghttp2 bindings was stronger at the start of the year, as Rust alternatives didn't implement the full spec. It seems they have caught up; at least now it's closer to what we could use. The only downside is that impls such as h2 advertise they depend on tokio, which is a non-starter for us as we'd like to use implementations that are built on futures / runtime agnostic. Maybe this is worth digging in deeper for though.

[yoshuawuyts]

I feel most points in this issue have already been addressed. I reckon we'd be better off migrating to smaller-grained issues from this point forward. Going to go ahead and close this, and then do some issue maintenance later this week. Thanks all!

[ibraheemdev]

Is there any update on HTTP/2 support? I couldn't fine a tracking issue for it...