Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add filename to result #78

Closed
wants to merge 11 commits into from
6,401 changes: 6,401 additions & 0 deletions package-lock.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
"lodash": "^4.13.1",
"node-resemble-js": "0.0.5",
"nodeclient-spectre": "^1.0.3",
"phantomjs-prebuilt": "^2.1.16",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this move to a dependency?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. I think this happened when I used yarn to add it. I will switch.

"platform": "^1.3.1",
"wdio-screenshot": "^0.6.0"
},
Expand All @@ -55,7 +56,6 @@
"mocha": "^2.4.5",
"nock": "^9.2.3",
"np": "^2.10.0",
"phantomjs": "^1.9.20",
"rimraf": "^2.5.2",
"sinon": "^1.17.4",
"wdio-mocha-framework": "^0.3.1",
Expand Down
3 changes: 2 additions & 1 deletion src/methods/BaseCompare.js
Original file line number Diff line number Diff line change
Expand Up @@ -126,8 +126,9 @@ export default class BaseCompare {
return path.join(runtimeConfigPath, `${name}-${suffix}.json`);
}

createResultReport(misMatchPercentage, isWithinMisMatchTolerance, isSameDimensions) {
createResultReport(filePaths, misMatchPercentage, isWithinMisMatchTolerance, isSameDimensions) {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chaning the order of arguments is a breaking change for all guys who uses custom compare methods. I would like to avoid this. Moreover this is to inflexibel and only useful for file comparison. I suggest to replace filePaths with an object that wil be assigned via Object.assign to the returned object. Then custom or other compare methods can enhance the report with additional information.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zinserjan Would you rather I made it the last argument, then? That way, if I am remembering my JS correctly, the value would just be null for those who aren't using the functionality?

return {
filePaths,
misMatchPercentage,
isWithinMisMatchTolerance,
isSameDimensions,
Expand Down
15 changes: 12 additions & 3 deletions src/methods/LocalCompare.js
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,13 @@ export default class LocalCompare extends BaseCompare {

const referenceExists = await fs.exists(referencePath);

var fileNames;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use const and assign value directly

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: extra line

fileNames = {
reference: referencePath,
actual: screenshotPath
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • add diff property with value of null to make sure the property is always there
  • replace key actual with screenshot to be consistent with the referenceName , screenshotName and diffName option.

};

if (referenceExists) {
log('reference exists, compare it with the taken now');
const captured = new Buffer(base64Screenshot, 'base64');
Expand All @@ -38,23 +45,25 @@ export default class LocalCompare extends BaseCompare {

const diffPath = this.getDiffFile(context);


Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: extra line

if (misMatchPercentage > misMatchTolerance) {
log(`Image is different! ${misMatchPercentage}%`);
const png = compareData.getDiffImage().pack();
await this.writeDiff(png, diffPath);
fileNames.diff = diffPath;

return this.createResultReport(misMatchPercentage, false, isSameDimensions);
return this.createResultReport(fileNames, misMatchPercentage, false, isSameDimensions);
} else {
log(`Image is within tolerance or the same`);
await fs.remove(diffPath);

return this.createResultReport(misMatchPercentage, true, isSameDimensions);
return this.createResultReport(fileNames, misMatchPercentage, true, isSameDimensions);
}

} else {
log('first run - create reference file');
await fs.outputFile(referencePath, base64Screenshot, 'base64');
return this.createResultReport(0, true, true);
return this.createResultReport(fileNames, 0, true, true);
}
}

Expand Down
6 changes: 5 additions & 1 deletion src/methods/SaveScreenshot.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,13 @@ export default class SaveScreenshot extends BaseCompare {
async processScreenshot(context, base64Screenshot) {
const screenshotPath = this.getScreenshotFile(context);

const fileNames = {
actual: screenshotPath
};

log(`create screenshot file at ${screenshotPath}`);
await fs.outputFile(screenshotPath, base64Screenshot, 'base64');
return this.createResultReport(0, true, true);
return this.createResultReport(fileNames, 0, true, true);
}

}
4 changes: 2 additions & 2 deletions src/methods/Spectre.js
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,11 @@ export default class Spectre extends BaseCompare {

if (result.pass) {
log(`${uploadName} - Image is within tolerance or the same`);
return this.createResultReport(result.diff, result.pass, true);
} else {
log(`${uploadName} - Image is different! ${result.diff}%`);
return this.createResultReport(result.diff, result.pass, true);
}

return this.createResultReport(null, result.diff, result.pass, true);
}

async onComplete() {
Expand Down
23 changes: 21 additions & 2 deletions test/unit/methods/LocalCompare.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ import LocalCompare from '../../../src/methods/LocalCompare';
const dirTmp = path.join(process.cwd(), '.tmp');
const dirFixture = path.join(__dirname, '../../fixture/');

const REFERENCE_SCREENSHOT = path.join(dirTmp, "reference.png");
const ACTUAL_SCREENSHOT = path.join(dirTmp, "screenshot.png");
const DIFF_SCREENSHOT = path.join(dirTmp, "diff.png");

async function readAsBase64(file) {
// read binary data
Expand All @@ -20,7 +23,6 @@ async function readAsBase64(file) {
}



describe('LocalCompare', function () {
beforeEach(async function () {
await fs.remove(dirTmp);
Expand Down Expand Up @@ -52,11 +54,23 @@ describe('LocalCompare', function () {
});

this.resultIdentical = {
filePaths: {
actual: ACTUAL_SCREENSHOT,
reference: REFERENCE_SCREENSHOT
},
misMatchPercentage: 0,
isWithinMisMatchTolerance: true,
isSameDimensions: true,
isExactSameImage: true
};

this.resultDiff = {
filePaths: {
actual: ACTUAL_SCREENSHOT,
reference: REFERENCE_SCREENSHOT,
diff: DIFF_SCREENSHOT
}
};
});

it('creates the captured screenshot', async function () {
Expand Down Expand Up @@ -175,7 +189,12 @@ describe('LocalCompare', function () {
assert.isTrue(existsDiff, 'Diff screenshot should exists');

// check if diff is correct
await compareImages(this.diffFile, path.join(dirFixture, 'image/100x100-diff.png'))
await compareImages(this.diffFile, path.join(dirFixture, 'image/100x100-diff.png'));



//check if returned filePaths correct
assert.deepEqual(this.resultDiff.filePaths, resultSecond.filePaths, 'Returned filepath should include diff')
});

it('deletes existing diff image when image is in tolerance now', async function () {
Expand Down
7 changes: 6 additions & 1 deletion test/unit/methods/SaveScreenshot.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import SaveScreenshot from '../../../src/methods/SaveScreenshot';
const dirTmp = path.join(process.cwd(), '.tmp');
const dirFixture = path.join(__dirname, '../../fixture/');

const ACTUAL_SCREENSHOT = path.join(dirTmp, "screenshot.png");

async function readAsBase64(file) {
// read binary data
Expand Down Expand Up @@ -40,19 +41,23 @@ describe('SaveScreenshot', function () {

context('processScreenshot', function () {
beforeEach(async function() {
this.referencFile = path.join(dirTmp, 'reference.png');
this.referencFile = path.join(dirTmp, 'screenshot.png');
this.getReferenceFile = stub().returns(this.referencFile);

this.saveScreenshot = new SaveScreenshot({
screenshotName: this.getReferenceFile,
});

this.resultIdentical = {
filePaths: {
actual: ACTUAL_SCREENSHOT
},
misMatchPercentage: 0,
isWithinMisMatchTolerance: true,
isSameDimensions: true,
isExactSameImage: true
};

});

it('creates a reference file for the first run', async function () {
Expand Down
7 changes: 6 additions & 1 deletion test/unit/methods/Spectre.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ import { createTestMethodInstance } from '../../helper/compareMethod';

const dirFixture = path.join(__dirname, '../../fixture/');

const BASE_PATH = "/Users/silne30/Desktop/wdio-visual-regression-service/.tmp/";
const REFERENCE_SCREENSHOT = BASE_PATH + "reference.png";
const ACTUAL_SCREENSHOT = BASE_PATH + "screenshot.png";

async function readAsBase64(file) {
// read binary data
Expand Down Expand Up @@ -112,9 +115,9 @@ describe('Spectre', function () {
});

const context = {};
const filePaths = null; //since no filepaths given

const resultIdentitical = await instance.processScreenshot(context, base64Screenshot1);

assert.strictEqual(this.getTest.callCount, 1, 'test getter should be called once');
assert.isTrue(this.getTest.calledWithExactly(context), 'test getter should receive context as arg');
assert.strictEqual(this.getBrowser.callCount, 1, 'browser getter should be called once');
Expand All @@ -123,6 +126,7 @@ describe('Spectre', function () {
assert.isTrue(this.getSize.calledWithExactly(context), 'size getter should receive context as arg');

assert.deepEqual(resultIdentitical, {
filePaths: filePaths,
misMatchPercentage: 0,
isWithinMisMatchTolerance: true,
isSameDimensions: true,
Expand Down Expand Up @@ -163,6 +167,7 @@ describe('Spectre', function () {
assert.isTrue(this.getSize.calledWithExactly(context), 'size getter should receive context as arg');

assert.deepEqual(resultDifferent, {
filePaths: filePaths,
misMatchPercentage: 4.56,
isWithinMisMatchTolerance: false,
isSameDimensions: true,
Expand Down
2 changes: 2 additions & 0 deletions test/wdio/VisualRegressionLauncher.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -231,12 +231,14 @@ describe('VisualRegressionLauncher - custom compare method & hooks', function ()

it('returns result from processScreenshot hook', async function () {
const expectedResult = {
filePaths: {},
misMatchPercentage: 10.05,
isWithinMisMatchTolerance: false,
isSameDimensions: true,
isExactSameImage: true,
};


this.processScreenshotStub.returns(expectedResult);

const options = {};
Expand Down
2 changes: 1 addition & 1 deletion test/wdio/wdio.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ exports.config = {
capabilities: [
{
browserName: 'phantomjs',
'phantomjs.binary.path': require('phantomjs').path,
'phantomjs.binary.path': require('phantomjs-prebuilt').path,
}
],
sync: false,
Expand Down
Loading