Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UI: HTTPAdapter #5637

Merged
merged 79 commits into from
Sep 4, 2019
Merged

UI: HTTPAdapter #5637

merged 79 commits into from
Sep 4, 2019

Conversation

johncowen
Copy link
Contributor

@johncowen johncowen commented Apr 10, 2019

This PR is a complete rewrite of the adapter data layer of the Consul UI.

Why

Briefly:

  1. We'd like access to the query object (get(this, 'store').query({theQueryObject})) at response time. Without having access to this we have to parse URLs and such like.
  2. The feature flag to enable non-deprecated code in 2.18 in the RESTAdapter and the features it enabled (that were required to enable what we needed to do) were removed in ~3.2. Continuing to use this will become a maintenance problem.
  3. Enable an API that can deal with pushed/partial data, we currently work around ember-data in order to implement our blocking query/pushed data support. It would be nice to be able to do new EventSource('/v1/internal/ui/services').addEventListener('message') within ember-data.
  4. Enable an injectable client. The 'client' in the RESTAdapter is more or less jQuery.ajax behind a private method. An injectable HTTP client means we can replace this at some point and opens many other possibilities such as native-like EventSources (via WebWorkers), streaming, caching etc.
  5. Massively simplify our data layer, to give an easy example as well as the diff below:
    KV Before: https://github.com/hashicorp/consul/blob/11362e77bebff2e2f66eb23822034d3d0d016bae/ui-v2/app/adapters/kv.js
    KV After: https://github.com/hashicorp/consul/blob/6d08cf13ede732f9f6248031546b6ec17c2b6014/ui-v2/app/adapters/kv.js
    The more 'nouns' we add to the UI the more of this complicated URL parsing code we have to add.

The important takeaway here is to consider how straightforward each individual Adapter is compared to what it was like previously. Considering the amount of individual Adapters is the thing that grows, its important that these are as simple as possible to create/maintain.

Other

Notes:

  1. All the tests either pass without changing, or we had to change the tests slightly (due to the reorg) but kept the coverage the same. There are 2 places where we change tests. 1 - as we now completely populate records when we receive a true using the query object we now have access to, and 2 was actually a bug in the tests that we found that wasn't effecting anything in the app.
  2. We've tried to completely replicate the RESTAdapter jQuery based client in the client/http request method for the moment.

There are still a few things left to do here, but I've had this hanging around a while and it's getting closer and closer to me actually needing this, so I've PRed it for 👀 while I finish up the following:

Todo:

  1. We'd like to slightly tweak the API so that unserialized and unserialized data is available to the requestFor... methods, currently only unserialized is. (see ui: Change signature of request/respond methods to take serialized and unserialized data #6285)
  2. Clean up client.request. Usually I wouldn't PR this like this, but the API is pretty much nailed down, we just need to refine the innards slightly and add a little unit testing including making sure headers are encoded etc.
  3. Possibly the odd nip and tuck elsewhere.
  4. Once approved I'll probably move some test files around some of them are now in the wrong place (what were previously adapter tests are now serializer tests), I've avoided moving these for the moment to try to make the already large set of changes easier to view. (see ui: HTTPAdapter - Move Tests #6281 ) Longer term, we could potentially remove a bundle of tests as a lot of the functionality is now massively simpler so its hardly worth testing (we'd be testing string interpolation - which we know works)

Lastly, there is probably a similar style serializer refactor coming down the line in a separate PR at some time following this.

@johncowen johncowen added the theme/ui Anything related to the UI label Apr 10, 2019
@johncowen johncowen requested a review from a team April 10, 2019 15:01
@johncowen johncowen force-pushed the feature/ui-http-adapter branch from 6d08cf1 to 408605b Compare April 29, 2019 11:36
@johncowen johncowen added this to the Upcoming milestone Apr 29, 2019
@johncowen johncowen force-pushed the ui-staging branch 2 times, most recently from 60bac11 to 9994267 Compare May 2, 2019 18:29
@johncowen johncowen force-pushed the feature/ui-http-adapter branch from 408605b to 1f51d18 Compare May 15, 2019 16:41
@johncowen johncowen force-pushed the feature/ui-http-adapter branch 2 times, most recently from af84d90 to 60007f6 Compare June 4, 2019 13:27
@johncowen johncowen marked this pull request as ready for review June 4, 2019 13:46
@johncowen johncowen removed this from the Upcoming milestone Jun 4, 2019
Copy link
Member

@hanshasselberg hanshasselberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skimmed over it and it looks good to me. There are many TODOs but I guess they are not meant to be done yet.

@johncowen
Copy link
Contributor Author

🙏 Thanks for the review!

Skimmed over it and it looks good to me. There are many TODOs but I guess they are not meant to be done yet.

Yup, exactly! As you can see this is a pretty big changeset so didn't want to keep going too far without someone taking a look. I'm gonna try and get this up to date and continue on here now. Probably another PR or 2 to come on top of here to finish up.

Thanks again!!

)

1. Adds a Listeners class, which lets us...
2. Add Listeners recursively. So you can
createListeners().add(createListeners())
3. Also add the ability to `.add` as a plain function

This moves the entire idea more towards a generic teardown utility
…5745)

* ui: Reconciliate ember-data store when records are deleted via blocking

Currently we are barely using the ember-data store/cache, but it will
still cache records in the store even though technically we aren't using
it.

This adds a SyncTime to every record that uses blocking queries so we
can delete older records from the ember-data cache to prevent them
building up

* ui: Add basic timestamp method we can access from tests, fixup tests

Adds a timestamp method that we can access from within tests so we can
test that the SyncTime is being set.

There is probably a better way to do this, but this is also probably the
simplest approach - we are also likely to revisit this at a later date
)

* ui: Allow text selection of clickable elements and their contents

This commit disables a click on mousedown be removing the `href`
attribute and moving it to a `data-href` attribute. On mouseup it will
only move it back if there is no selection. This means that an anchor
will only be followed on click _if_ there is no selection.

This fixes the fact that whenever you select some copy within a
clickable element it immediately throws you into the linked page when
you release your mouse.

Further notes:

We use the `isCollapsed` property here which 'seems' to be classed as
'experimental' in one place where I researched it:

https://developer.mozilla.org/en-US/docs/Web/API/Selection/isCollapsed

Although in others it makes no mention of this 'experimental' e.g:

- https://webplatform.github.io/docs/dom/Selection/isCollapsed/
- https://w3c.github.io/selection-api/#dom-selection-iscollapsed

I may have gone a little overboard in feature detection for this, but I
conscious of that fact that if `isCollapsed` doesn't exist at some point
in the future (something that seems unlikely). The code here will have
no effect on the UI. But I'd specifically like a second pair of eyes on
that.

* ui: Don't break right click, detects a secondary click on mousedown

* ui: Put anchor selection capability behind an ENV var
…5703)

EventSources will wait for 1 tick before 'opening'. There is always the
chance that the EventSource is '.close()'ed before that tick. We
therefore check the 'readyState' before opening the EventSource
…icons` (#6021)

1. Rebuild the heathchecked-resource component now we can copy and paste
2. As the above rebuild came with new icons, we also swapped out 'most'
of the other areas where we were using these new icons, plus any icons
that were effected by the new icon placeholders
3. Begin to remove more and more of the project specific icons (now
replaced by the shared ones)
ember-cli-api-double has been upgraded for 3 things:

1. Use the correct configuration flags
2. Automatically include the necessary files to enable the api doubles
without requiring a server. This can be disabled to provide custom
functionality (so we can stitch our BDD style testing in with this)
3. When used statically, read the cookies from the users browser to
enable basic ad-hoc double editing (e.g. CONSUL_SERVICE_COUNT=100000)

Once upgraded we've now moved the config to the correct place, added a
new environment (staging) to use the static-style of doubles
The test environment continues to use custom cookie setting and url
checking so we disable this 'auto importing' by setting 'auto-import' to
false for the configuration for the addon.

We also added a couple of new package script targets to explicitly serve
or build the UI with the entirely static UI.
Initialize session value to `null` to prevent stickiness from a session the previous KV
-Enable blocking queries by default
-Change assertion to check for the last PUT request, not just any request for session destruction from a node page.

Since we've now turned on blocking queries by default this means that a
second GET request is made after the PUT request that we are asserting
for but before the assertion itself, this meant the assertion failed. We
double checked this by turning off blocking queries for this test using

```
And settings from yaml
---
consul:client:
  blocking: 0
---
```

which made the test pass again.

As moving forwards blocking queries will be on by default, we didn't
want to disable blocking queries for this test, so we now assert the
last PUT request specifically. This means we continue to assert that the
session has been destroyed but means we don't get into problems of
ordering of requests here
Right now we only use CONSUL_DOCS_URL, the others are for future usage
- yarn upgrade consul-api-double which includes `status/leader`
- add all the ember-data things required to call a new endpoint
- Pass the new leader variable through to the template
- use the new leader variable in the template to set a leader
- add acceptance testing to verify leaders are highlighted
- Change testing navigation/api requests to status/leader (on the node listing page, status/leader is now the last get request to
be called).
- Template whitespace commit (less indenting)
- adds a test to to assert no errors happen with an unelected leader
…tc (#6034)

This adds the component but doesn't yet use it anywhere. No tests
are added here as there isn't an awful lot to test.
- Add netlify redirects file
- Add netlify makefile target and commented config settings
@johncowen
Copy link
Contributor Author

Furthermore as a result of trying to keep this branch up to date with ui-staging and master the merge has duplicated some code here - we'll try to resolve this by adding a commit or 2 to the end of #6386 as its related to where the duplicated code seems to have appeared.

* Migrate queryLeader to new HTTPAdapter
* Add some extra tests for queryLeader
@johncowen
Copy link
Contributor Author

Tests are passing again here now so this could be merged to ui-staging and then down to master when we are ready to do that.

Future work at some point in the future (rough notes):

  1. Finish of the similar serializer refactor based on Response/{header, body} shaped responses (similar to https://developer.mozilla.org/en-US/docs/Web/API/Response) instead of just the body. This wouldn't have to use a Response object just be a pojo with similar prop names.
  2. The client here is a close copy of the ember-data RESTAdapter jQuery based client (in an attempt to keep things as close to the original behaviour as possible). We were going to refactor this out a little more, but as we would like to move to something either fetch based or WebWorker based in the near term - what would probably be wasted effort. We may as well just go straight for WebWorker and/or fetch.

We've bumped this to 'future work' party due to the long running of this PR series, and partly due to the fact that there is enough work done here to allow us to upgrade to a more up to date version of ember.

@johncowen johncowen merged commit 4b347cb into ui-staging Sep 4, 2019
@johncowen johncowen deleted the feature/ui-http-adapter branch September 12, 2019 12:38
johncowen added a commit that referenced this pull request Sep 23, 2019
johncowen added a commit that referenced this pull request Sep 25, 2019
johncowen added a commit that referenced this pull request Sep 30, 2019
johncowen added a commit that referenced this pull request Sep 30, 2019
## HTTPAdapter (#5637)

## Ember upgrade 2.18 > 3.12 (#6448)

### Proxies can no longer get away with not calling _super

This means that we can't use create anymore to define dynamic methods.
Therefore we dynamically make 2 extended Proxies on demand, and then
create from those. Therefore we can call _super in the init method of
the extended Proxies.

### We aren't allowed to reset a service anymore

We never actually need to now anyway, this is a remnant of the refactor
from browser based confirmations. We fix it as simply as possible here
but will revisit and remove the old browser confirm functionality at a
later date

### Revert classes to use ES5 style to workaround babel transp. probs

Using a mixture of ES6 classes (and hence super) and arrow functions
means that when babel transpiles the arrow functions down to ES5, a
reference to this is moved before the call to super, hence causing a js
error.

Furthermore, we the testing environment no longer lets use use
apply/call on the constructor.

These errors only manifests during testing (only in the testing
environment), the application itself runs fine with no problems without
this change.

Using ES5 style class definitions give us freedom to do all of the above
without causing any errors, so we reverted these classes back to ES5
class definitions

### Skip test that seems to have changed due to a change in RSVP timing

This test tests a usecase/area of the API that will probably never ever
be used, it was more testing out the API. We've skipped the test for now
as this doesn't affect the application itself, but left a note to come
back here later to investigate further

### Remove enumerableContentDidChange

Initial testing looks like we don't need to call this function anymore,
the function no longer exists

### Rework Changeset.isSaving to take into account new ember APIs

Setting/hanging a computedProperty of an instantiated object no longer
works. Move to setting it on the prototype/class definition instead

### Change how we detect whether something requires listening

New ember API's have changed how you can detect whether something is a
computedProperty or not. It's not immediately clear if its even possible
now. Therefore we change how we detect whether something should be
listened to or not by just looking for presence of `addEventListener`

### Potentially temporary change of ci test scripts to ensure deps exist

All our tooling scripts run through a Makefile (for people familiar with
only using those), which then call yarn scripts which can be called
independently (for people familar with only using yarn).

The Makefile targets always check to make sure all the dependencies are
installed before running anything that requires them (building, testing
etc).

The CI scripts/targets didn't follow this same route and called the yarn
scripts directly (usually CI builds a cache of the dependencies first).

For some reason this cache isn't doing what it usually does, and it
looks as though, in CI, ember isn't installed.

This commit makes the CI scripts consistently use the same method as all
of the other tooling scripts (Makefile target > Install Deps if
required > call yarn script). This should install the dependencies if
for some reason the CI cache building doesn't complete/isn't successful.

Potentially this commit may be reverted if, the root of the problem is
elsewhere, although consistency is always good, so it might be a good
idea to leave this commit as is even if we need to debug and fix things
elsewhere.

### Make test-parallel consistent with the rest of the tooling scripts

As we are here making changes for CI purposes (making test-ci
consistent), we spotted that test-parallel is also inconsistent and also
the README manual instructions won't work without `ember` installed
globally.

This commit makes everything consistent and changes the manual
instructions to use the local ember instance that gets installed via
yarn

### Re-wrangle catchable to fit with new ember 3.12 APIs

In the upgrade from ember 3.8 > 3.12 the public interfaces for
ComputedProperties have changed slightly. `meta` is no longer a public
property of ComputedProperty but of a ComputedDecoratorImpl mixin
instead.

https://github.com/emberjs/ember.js/blob/7e4ba1096e3c2e3e0dde186d5ca52ff19cb8720a/packages/%40ember/-internals/metal/lib/computed.ts#L725

There seems to be no way, by just using publically available
methods, to replicate this behaviour so that we can create our own
'ComputedProperty` factory via injecting the ComputedProperty class as
we did previously.

https://github.com/hashicorp/consul/blob/3f333bada181aaf6340523ca2268a28d1a7db214/ui-v2/app/utils/computed/factory.js#L1-L18

Instead we dynamically hang our `Catchable` `catch` method off the
instantiated ComputedProperty. In doing it like this `ComputedProperty`
has already has its `meta` method mixed in so we don't have to manually
mix it in ourselves (which doesn't seem possible)

This functionality is only used during our work in trying to ensure
our EventSource/BlockingQuery work was as 'ember-like' as possible (i.e.
using the traditional Route.model hooks and ember-like Controller
properties). Our ongoing/upcoming work on a componentized approach to
data a.k.a `<DataSource />` means we will be able to remove the majority
of the code involved here now that it seems to be under an amount of
flux in ember.

### Build bindata_assetfs.go with new UI changes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
theme/ui Anything related to the UI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants