Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check functionality #552

Merged
merged 1 commit into from
Apr 10, 2018
Merged

Check functionality #552

merged 1 commit into from
Apr 10, 2018

Conversation

RandomSeeded
Copy link

No description provided.

@RandomSeeded
Copy link
Author

First draft of PR to resolve #551

Please LMK if this is along the lines of what you're looking for. I'll also try to look into how to test this later; I've never used lab before.


internals.migrationsDir = migrator.migrationsDir;

migrator.driver.createMigrationsTable(function (err) {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to me like we could nuke the createMigrationsTable call; but it also seems to me like we could've done the same for dry-run with migrations up, so I left in place for parity

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As soon as we read from the db, we currently need to create it, to avoid erroring out.

@@ -144,8 +146,13 @@ module.exports = function (internals, isModule) {
internals.notransactions = internals.argv['non-transactional'];
internals.dryRun = internals.argv['dry-run'];
global.dryRun = internals.dryRun;
internals.check = internals.argv['check'];
global.check = internals.check;
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually maybe not necessary 🤔

why is it done for dryRun?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is from ancient times ;) Don't add any new global flags though, the old ones will disappear and are only still there until they get deprecated. and removed

lib/migrator.js Outdated
@@ -119,6 +119,45 @@ Migrator.prototype = {
}
},

check: function (funcOrOpts, callback) {
var self = this;
// Not gonna lie dont really know whats up with the funcOrOpts pattern, this is just c/p'd
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ping on this

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you do not execute anything, you wont need it. The new loader is actually way more understandable:

https://github.com/db-migrate/node-db-migrate/pull/537/files#diff-69369e6549a54103e4d5c8eb543712c1

The funcOrOpts basically is an acceptor to directly execute a migration and the same time it is the enter callback for normal migration behavior.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So just remove that bit here

@RandomSeeded
Copy link
Author

modification to db-migrate-shared: db-migrate/shared#4

@wzrdtales
Copy link
Member

Am on the road now again, will have a look later again, I do highly appreciate your effort and your valuable contribution to the open source community 🎉

lib/migrator.js Outdated
@@ -119,6 +119,45 @@ Migrator.prototype = {
}
},

check: function (funcOrOpts, callback) {
var self = this;
// Not gonna lie dont really know whats up with the funcOrOpts pattern, this is just c/p'd
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So just remove that bit here

lib/migrator.js Outdated
@@ -180,6 +219,12 @@ Migrator.prototype = {
return;
}

if (self.internals.check) {
log.info('Migrations to run:', toRun.map(migration => migration.name));
callback(null);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is probably more worthwhile to not only log it, but to deliver the actual list for the programmatic mode.

lib/migrator.js Outdated
@@ -234,6 +279,12 @@ Migrator.prototype = {
return;
}

if (self.internals.check) {
log.info('Migrations to run:', toRun.map(migration => migration.name));
callback(null);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

@RandomSeeded
Copy link
Author

Thanks for the review, I'll try to get to it this upcoming week :)

@RandomSeeded
Copy link
Author

@wzrdtales back to you :)

@wzrdtales
Copy link
Member

Looks good will do the complete review tomorrow. Please note that you also will have to update your commit messages to comply to the standards and add the DCO. Don't know why the bots did not activated on your branch.

@RandomSeeded RandomSeeded force-pushed the nate/check branch 2 times, most recently from 44b1421 to 093993b Compare March 27, 2018 02:38
@RandomSeeded
Copy link
Author

looks like dco check is now running and passing

@wzrdtales
Copy link
Member

Yeah the commit lint is dead though, strange will have a look at this, thanks for revisiting your commits already though :)

@wzrdtales
Copy link
Member

Looks good so far, but tests are missing.

@RandomSeeded
Copy link
Author

Will do early next week

@RandomSeeded
Copy link
Author

@wzrdtales added to api.js and also added a test to api_test.js

Looked through the test dir pretty extensively to try to see how up, down, and dryRun were tested today to base further tests off of and couldn't really find anything 🤷‍♂️

Copy link
Member

@wzrdtales wzrdtales left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test for check method would be perfect.

I will take on the rest for you.

@@ -119,6 +119,40 @@ Migrator.prototype = {
}
},

check: function (funcOrOpts, callback) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

up and down was implicitly covered by the driver tests, which have been moved together with their drivers. So you're right, that these need to be created as well, not by you though :). However, if you could add a unit test for this function, validating that it returns the expected result for a given input. You can mock away loadFromFileSystem and loadFromDatabase, those are already proven to work.

@commitlint-wzrdtales
Copy link

There were the following issues with this Pull Request

  • Commit: ce28744
    • ✖ message may not be empty
      , - ✖ type may not be empty

You may need to change the commit messages to comply with the repository contributing guidelines.


🤖 This comment was generated by commitlint[bot]. Please report issues here.

Happy coding!

@RandomSeeded
Copy link
Author

Hey again @wzrdtales , pushed a commit with a test. Didn't bother to squash yet, because before merging we will also still need to do the following:

I would expect the new test to fail CI until ^ is merged, but it does pass on my local with the packages linked (or feel free to repro it passing by pointing at the github hash).

Thanks for being so prompt at getting back to me on this, it's really impressive :)

@wzrdtales
Copy link
Member

I will review this tomorrow when I got a bit of sleep, too tired right now. Thanks for your efforts! :)

@wzrdtales
Copy link
Member

Looks good, will merge when you comply here to the commitlint and dco and on the other side remove the version bump :)

@RandomSeeded
Copy link
Author

RandomSeeded commented Apr 9, 2018

@wzrdtales removed version bump from db-migrate-shared and passing DCO. I'd expect that we'd also want to update this package.json to reflect the new version of db-migrate-shared but for now waiting til that merges. I can prematurely update the dependency if you'd like, too, either is fine

@wzrdtales
Copy link
Member

Merged in and updated on the master branch.

@wzrdtales
Copy link
Member

Rebasing against master should make the tests pass now.

Signed-off-by: RandomSeeded <nate@blend.com>
@RandomSeeded
Copy link
Author

@wzrdtales done

@wzrdtales wzrdtales merged commit 0c19dba into db-migrate:master Apr 10, 2018
@wzrdtales wzrdtales mentioned this pull request Apr 10, 2018
9 tasks
wzrdtales added a commit that referenced this pull request Apr 10, 2018
#552 introduced a regression and effectively disfunctioned the reset function, since it
did append just to the actionables within the run context.

It was moved into its own context.

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>
wzrdtales added a commit that referenced this pull request Apr 10, 2018
#552 introduced a regression and effectively disfunctioned the reset function, since it
did append just to the actionables within the run context.

It was moved into its own context.

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>
wzrdtales added a commit that referenced this pull request Apr 10, 2018
#552 introduced a regression and effectively disfunctioned the reset function, since it
did append just to the actionables within the run context.

It was moved into its own context.

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>
RandomSeeded added a commit to blend/node-db-migrate that referenced this pull request May 29, 2018
* cs fixes

* add concepting to comments

* fix driver inclusion

* remove sqlite dependency, add sqlite to devdependencies and outsource driver

* remove unnecessary files

* remove mongodb dependency, outsource mongodb driver

* Bump sqlite3 package version to fix issue with node 4.0.0

* Bump sqlite3 package version to fix issue with node 4.0.0

* add node version 4.0, remove iojs allowed build failures

* remove mysql dependency and add to dev for testing purposes

* fix index_test tests

* change base test to extern base

* fix index test

* remove last driver dependencies

resolves db-migrate#227

* add pg to dev dependencies

* ?

* updated fixed pg driver

* add error message to executeDB and handle callback

* return error message if available or the object/message directly passed through

* fix some cs issues

* add missing nosql methods to interface

* do not strip private functions from driver

removed unnecessary entries from migrator interface
do not strip any `_function` unless it is explicitly defined via exclude

* change config to internal scope

* fix test for new internal config

* fix last details to move config i into internal scope

* remove debug message

* remove es5 from jshint rules, fix one cs

*  add rules to jshint

* fix some cs issues and adjust rules

* adjust some settings and add jscs already

* [cs] fixed some issues and do some cleanup

* update lab

* update dependency and check

* add 4.1 to travis ci

* change couple of templates

* update sql templates

* add undo seed function to api

* fix bug on up/down of seeders

* add reset method and limit down methods of seeds

* add dummy calls to linked seeders and migrations

* link to read the docs and remove readme documentation

* 0.10.0-beta.1

* remove the badge ending from the documentation url

* make entry in readme more clear to be also a user manual

* Extend startup with logging and failure handling on 0.9.x versions

Before this patch users may ran into a local installation older than 0.9.x, this patch
ensures that the version is compatible to be preffered over the global version. If the
local version is to old (<0.10.0) the global version will be used instead. If this is
the case a warn message gets displayed to the user.

Additionally added informational logging within the verbose mode if a local version gets
preferred.

solves db-migrate#317

* 0.10.0-beta.2

* add lts node.js version to travis tests

* Resolve driver from local projects

Fixes db-migrate#318

* use synchronized variant of resolve for drivers

* 0.10.0-beta.3

* fix sql templates to work with 0.10.x again

* ignore on init formatting

* 0.10.0-beta.4

* Support setting default environment via ENV variable.

* Support setting default environment via ENV variable.

* coffeescript template refactory

coffeescript template refactory

by default coffeescript always returns the last line/variable, so there is no need to use the ~return~ keyword, there is not need for ; also.

* add scope logging

* [bugfix] internal linking causes infinite loop

fixes db-migrate#329

* add scope logging

* [bugfix] internal linking causes infinite loop

fixes db-migrate#329

* coffeescript template refactory

coffeescript template refactory

by default coffeescript always returns the last line/variable, so there is no need to use the ~return~ keyword, there is not need for ; also.

* 0.10.0-beta.5

* remove debug message

* 0.10.0-beta.6

* allow configuration through options object

* adjust test

* add direct execution of seeders and migrator links

* Fix config file setting.

* update parse db-url to 0.3.0

Introduces mongodb uri support

* update some dependencies

* fix(migration):

A promise is not properly identified and returning an empty resolve ends up in
db-migrate hanging up forever.

Fixes db-migrate#343

* fix(api): callback not called on run

If db-mgirate is run via any run method (except directly from console), the callback was not available
to the migration functions anymore.

* fix(api): callback called twice

Revert the change to add the callback if it is undefined.

Fixes db-migrate#343

* Fix promise issue for down migrations.

* refactor(migration): Make check for promise more reliable

* fix(test): Stub MySQL connect method instead of calling the original

The MySQL driver now connects directly to the driver, instead on the first query. This
caused some trouble with the ssh tunnel test, this functionality was stubbed.

Fixes db-migrate#348

* chore(security): Bump dependency because of security vulnerability

* pull down package.json

* chore(dependencies): update and remove unneeded dependencies

* Update data_type.js

For consistency :-)

* chore(travis): change travis config to use modern compiler

* chore(travis): add node 5 to ci builds

* chore(travis): remove obsolete node 4.1

Keep 4.2 lts and node 4 in the build which is usually enough.

* Avoid warning in promise/callback bridge code

When Bluebird detects when a promise is created within a promise
handler, but is not returned, it [emits a warning][warn] 'a promise was
created in a handler but was not returned from it'.

This happens if the `callback()` function is implemented using Promises,
but uses `nodeify()` to make it callback compatible. By using nodeify
here, it handles the returned Promise (if any) and avoids the warning.

 [warn]: https://github.com/petkaantonov/bluebird/blob/master/docs/docs/warning-explanations.md

* Updated documentation link in README

Current documentation link points to a 404 page, updated it with a working version.

* chore(travis): add node 6

* fix(api): fix race condition on create migration

A callback was placed not inside but after the callback which it belongs to. This
resulted in a very rarely occuring race condition when creating migrations without
an existing migration directory.

Fixes db-migrate#376

* 0.10.0-beta.12

* chore(compatibility): update dependencies

* chore(dev): add .tern file to project

* refactor(log): move log into new extern lib

Refers to db-migrate#382

* refactor(datatype): move dataType into new extern lib

Refers to db-migrate#382

* refactor(remove): remove log and dataType artifacts

Refers to db-migrate#382

* refactor(util): remove util artifact

Resolves db-migrate#382

* detect real callback for several api functions

* feat(api): promisify all current api methods

* refactor(ssh): delay loading of tunnel-ssh and improve load time

This improves the loadtime of db-migrate by 47%. To be exact from ~22ms to ~15ms.

Resolves db-migrate#387

* refactor(modularize): require inflection from npm instead

* refactor(tests): added shadow test and rewrite api test

* chore(comments): add some notes to the api tests

* refactor(globals): remove several globals, this might break things

There are several things that are exposed as global. For example Migrator, Seeder, Class, or the async and dbm globals.

Most probably finally removing async and dbm which have been marked deprecated, will end up for some people in a breaking change. But we need to remove all those
globals, to not interfere with projects using db-migrate. DB-Migrate is not just a cli module anymore though. In this step we now also finally removed internals completely
from the API, and needed to adjust the onComplete callback to actually do this. This change could also lead to a breaking change and should thus be communicated

All in all this is a big clean up and also two not necessary functions were removed.

* chore(parallelize): parallelize non interferring api tests

* refactor(tests): add index tests and some configs

* refactor(tests): refactored util_tests

* refactor(tests): refactored migration tests

* refactor(tests): refactored config tests

* refactor(tests): refactored base tests

* refactor(tests): add create tests and remove sample shadows

* chore(ci,dep): update ci and dependencies

* chore(test): create Makefile for tests and edit ci config

Removing 0.10 and 0.12 from tests

* chore(ci): clean up ci config

* chore(dependencies): upgrade dotenv and pkginfo

* chore(devdep): update and remove some devDependencies

* use path.join (db-migrate#353)

refactor(template): use path.join as it is intended to be used

* fix(template): fix unnoticed error introduced in the last merge request

Wrong migrations have been generated

* feat(plugins): add basic support for plugins and improve performance

Some libraries have been changed to be delay loaded. DB-Migrate is back to the minimal load time possible again, if no command at all is entered.

Basic plugin support and the first hook, in this case of the type overwrite has been added. An functional example of a yaml config plugin
has been published.
Also some old functions have been cleaned up which are not necessary any more.

refers to db-migrate#397

* chore(leftover): remove example plugin leftover from package.json

* fix(api): fix introduced undefined behavior of specified configs

internal.cwd added again, which is needed.

* fix(api): fix scoping

The api was missing the matching internal, which resulted in a missing prefix
when executing scope migrations.

fixes db-migrate#409

* most of the first protocol transition and the transition helper
is implemented already. Just the logic to actually call the helper is currently missing.

* feat(transitioner): add transitioner to easen the process of protocol changes

Resolves db-migrate#403

* feat(hook): parser hook and transitioner api

Added a hook for parsers and added the API endpoints for the transitioner.

Refers to db-migrate#403
Refers to db-migrate#397

* fix(config): Don't throw if environment variable is empty

Fixes db-migrate#411

* chore(hook): allow also to hook without adding any extension

* fix(transitioner): add new parser internal to transitioner

* fix(transitioner): catch whitespaces properly

* chore(templates): add meta protocol v1 to already aligned migration templates

* feat(plugin): add basic plugin support

Added basic plugin support, enabling to inject at multiple targets of the API, config and register
pre compiler.

Resolves db-migrate#397
Refers db-migrate#396

* fix(plugin): use correct path to include plugins

* fix(errorhandling): Add missing error assertion in executeDB

fixes db-migrate#381

* feat(sync): add sync mode

Refers to db-migrate#383
Resolves db-migrate#313
Refers to db-migrate#222

* refactor(tests): remove tests from main repo

Tests have been moved to their driver repo.

Resolves db-migrate#399

* refactor(cleanup): cleanup package.json and add db.config.ci back

* chore(ci): exclude interfaces

* chore(ci): Readd missing dependencies

* chore(test): Add Test for privateKey param on shh tunnel

* fix(resolve): Check if resolved version has plugin support

fix db-migrate#425

* log error on undefined stack

* refactor(api): cleanup information and added commentary

* feat(config): add rc style configs

fixes db-migrate#308
fixes db-migrate#406

* refactor(api): refactor tests and make api.js more modular

* fix(api): add missing reference to sync

* refactor(api): remove last output of postponed seeders

* add test dummies for api

* add first programable api tests

* directly use promises in tests

* Double quoting interpolated file name for coffee

* fix(create): use same timestamp in every created file

* fix(args): dont parse when called as module

fixes db-migrate#449

* fix deep config replacements

fixes db-migrate#473

* feat(config): helper to overwrite and extend configuration

This allows to parse a url from an url property and as well to overwrite and extend
variables as needed.
To pass in url as normal value an object is needed like the following: { url: { value: "http://example.com } }.

Fixes db-migrate#349
Fixes db-migrate/pg#8
Fixes db-migrate#488
Fixes db-migrate#463

* fix(tests): fix breaking tests

* refactor(config): do not just interpret database.json

also check for either a passed config object OR a config file exists

* fix(create): Fix create when using db-migrate as module

Bug introduced on db-migrate#485

Fixes: db-migrate#493

* test(create): Fix testing as module when no database.json on root

* fix small typo in cli help menu

* allow rc file to contain location of config file

* print the reason in unhandledRejection evnet if reason.stack does not exist

* complete refactor of api js

* add eslint to dependencies

* fix a few bugs and update tests

* fix last tests

* adjust to cs

* adjust manual changes for cs

* remove process.exit when no callback passed and remove optimist help when in module mode

Fixes db-migrate#516

* add changelog and release 0.10.0

* 0.10.0

* adjusted tests and updated devDependencies

Refers to db-migrate#518

* readd pkg-lock

* change tested versions

* rewrite latest to node

* add linting and fix cs on config test

* add commitlint

* remove parallel: true as it was deprecated

* add stale config

* use more appropiate label

* Create CODE_OF_CONDUCT.md

* fix(db): wrong reference to connect causes db:create to fail

Fixes db-migrate#520

* 0.10.1

* add changelog

* fix(log): error ended up in unreadable errors

Fixes db-migrate#524
Fixes db-migrate#521

* add changelog

* chore: Remove `moment`.

Signed-off-by: wtgtybhertgeghgtwtg <wtgtybhertgeghgtwtg@gmail.com>

* feat(issuetemplate): added a github issue template

Signed-off-by: BorntraegerMarc <marc.borntraeger@gmail.com>

* fix(exitCode): wrong check for existence fixed

A wrong check for existence resulted in events not
being loaded on version sized like 4.0.0. This resulted
in non zero error codes when actually running into errors.

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* refactor(deprecationNotice): remove insert deprecation

This temporarily removes the note being displayed to the user
when an insert call is being made. This will be removed until
seeders really going to be introduced.

Refers to db-migrate#215

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* fix(exitCode): wrong exit code on db methods

DB Calls always returned an exit code of 0, which leads
to an unexpected behavior on the user side.

Fixes db-migrate#534

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* fix(switchDatabase): no error was thrown on scope switch

switchDatabase error was unhandled, which resulted in an unhandled
scope switch error.

Fixes db-migrate#470

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* 0.10.3

* add changelog

* fix(insert): add missing insert entry to interface

Fixes db-migrate#542

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* 0.10.4

* changelog

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* 0.10.5

* update to use real checkboxes

* Update ISSUE_TEMPLATE.md

* feat(contribution): enrich contribution instructions

Refers to db-migrate#549

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* feat(contribution): enrich contribution instructions, issues

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* feat(progamableApi): CMD options can be passed programatically now

Signed-off-by: Marc Bornträger <marc.borntraeger@gmail.com>

* fix(ci): add ignores for backported features

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* chore: update dependencies

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* feat(progamableApi): using const now

Signed-off-by: BorntraegerMarc <marc.borntraeger@gmail.com>

* add changelog

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* 0.10.6

* chore: add test to detect cmdOptions functionality

Refers to db-migrate#560
Refers to db-migrate#557

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* fix(progamableApi): cmdOptions get passed into setDefaultArgv now

Signed-off-by: Marc Bornträger <marc.borntraeger@gmail.com>

* 0.10.7

* add changelog

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* chore: update dependency

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* chore: update dependency

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* feat(check): add check functionality to determine migrations to run

Signed-off-by: RandomSeeded <nate@blend.com>

* 0.11.0

* add changelog

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* fix(reset): regression introduced in check functionality

db-migrate#552 introduced a regression and effectively disfunctioned the reset function, since it
did append just to the actionables within the run context.

It was moved into its own context.

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* add changelog

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* 0.11.1

* fix(check): fix check via API not passing results to the callback

Signed-off-by: RandomSeeded <nate@blend.com>

* save

* save

* updates

* cleanup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants