Calling `pending` in a spec marks the spec as failed. #2454

sveneh · 2015-08-27T10:09:33Z

See the attached testcase

https://gist.github.com/sveneh/9c589d321e6fc31f7460

When I run this in protractor@2.1.0 using jasmine@2.3.1 I get this:

>protractor pending.conf.js
Starting selenium standalone server...
[launcher] Running 1 instances of WebDriver
Selenium standalone server started at http://192.168.88.100:60036/wd/hub
Started
F*

Failures:
1) a spec with pending should be pending
  Message:
    Failed: => marked PendingThis is pending.
  Stack:
    Error: Failed: => marked PendingThis is pending.
Pending:

1) a spec with pending should be also pending
  No reason given

2 specs, 1 failure, 1 pending spec
Finished in 0.006 seconds
Shutting down selenium standalone server.
[launcher] 0 instance(s) of WebDriver still running
[launcher] chrome #1 failed 1 test(s)
[launcher] overall: 1 failed spec(s)
[launcher] Process exited with error code 1

running the same spec in pure jasmine@2.3.1:

>jasmine 
Started
**

Pending:

1) a spec with pending should be pending
  This is pending.


2) a spec with pending should be also pending
  No reason given

2 specs, 0 failures, 2 pending specs
Finished in 0.002 seconds

Protractor should not fail specs that are marked as pending.

The text was updated successfully, but these errors were encountered:

cstraynor · 2015-09-08T15:54:52Z

I'm experiencing this too. It would be good to get resolved so I can have accurate reports to delineate tests actually failing and tests being skipped.

theandrewlane · 2015-09-14T16:56:21Z

Someone please fix this!! In the meantime, to mark as pending without using pending("reason") use xit('some spec description', function (){

lynchblue · 2015-09-15T16:05:33Z

+1 That's a pretty cool jasmine2 feature and it's a pity that we cannot use it with protractor. Is anyone taking care of this?

ctapobep · 2015-09-20T16:50:06Z

+1

juliemr · 2015-10-02T00:15:03Z

See angular/jasminewd#27 (comment)

bruderol · 2016-03-18T15:19:15Z

+1 this is really anoying! I want to be able to put tests to pending with correct messages in log.

bruderol · 2016-03-18T16:23:34Z

As a workaround I introduced an own pending to use in my protractor tests (overwriting the jasmine pending), maybe that is helpful for somebody else as a workaround:

            // in my protractor onPrepare code:
            var _pending = pending;
            pending = function(pendingMessage) {

                return {

                    it: function(description, itFunction) {
                        var consoleColorYellowStart = '\u001B[33m';
                        var consoleColorYellowEnd = '\u001B[0m';
                        return xit(description + '\n  ' + consoleColorYellowStart + 'Marked as Pending: ' + pendingMessage + consoleColorYellowEnd, itFunction);
                    },

                    describe: function(description, describeFunction) {
                        return describe(description, function() {
                            _pending(pendingMessage);
                            return describeFunction();
                        });
                    },

                    here: function() {
                        return _pending(pendingMessage);
                    }

                };

            };

Usage:
pending('my pending comment').describe('my describe block', function() { ... });
or
pending('my pending comment').it('my it block', function() { ... });
or (just in case you still need the old pure jasmine pending implementation somewhere else, with all advantages and disadvantages)
pending('my pending comment').here();

akstatic · 2016-03-30T04:46:26Z

+1

lgarest · 2016-04-13T15:23:20Z

Still experiencing this, meanwhile the workaround works like a charm @bruderol thanks!

moneytree-doug · 2016-04-15T09:21:51Z

Any updates on this issue?

mattc41190 · 2016-04-25T20:39:50Z

Also feeling some burn here +1

rcbop · 2016-04-29T03:37:26Z

+1 also happening here. :(

d-werner · 2016-04-29T14:12:59Z

+1

ArturKwiatkowski · 2016-05-05T18:07:59Z

+1

bruderol · 2016-05-05T18:45:28Z

In the meantime I found out that there is a possibility to set comments on tests ignored with xit and therefore even improved my workaround a little bit:

            // in my protractor onPrepare code:
            var _pending = pending;
            pending = function(pendingMessage) {

                return {

                    it: function(description, itFunction) {
                         var spec = xit(desription, itFunction);
                         spec.pend(pendingMessage);
                         return spec;                        
                    },

                    describe: function(description, describeFunction) {
                        return describe(description, function() {
                            _pending(pendingMessage);
                            return describeFunction();
                        });
                    },

                    here: function() {
                        return _pending(pendingMessage);
                    }

                };

            };

Usage:
pending('my pending comment').describe('my describe block', function() { ... });
or
pending('my pending comment').it('my it block', function() { ... });
or (just in case you still need the old pure jasmine pending implementation somewhere else, with all advantages and disadvantages)
pending('my pending comment').here();

stheikki · 2016-06-02T22:56:29Z

also experiencing this, is this issue currently being worked on?

dverbovyi · 2016-06-08T16:55:51Z

+1. Also faced with this issue

HugoDoyon · 2016-07-12T16:09:50Z

+1 Should be priorize, this is a very usefull feature.

juliemr · 2016-07-14T18:49:50Z

Closing in favor of angular/jasminewd#32

Note that this is pending a change to Jasmine.

sgibson21 · 2017-02-28T17:42:09Z

I extended the jasmine-spec-reporter (https://www.npmjs.com/package/jasmine-spec-reporter)

var SpecReporter = require('jasmine-spec-reporter');
var PendingSpecReporter = SpecReporter;

PendingSpecReporter.prototype.specDone = function (spec) {
  this.metrics.stopSpec(spec);
  var pending = false;
  for(var i = 0; i < spec.failedExpectations.length; i++) {
    if ( spec.failedExpectations[i].message.toLowerCase().indexOf('pending') >= 0) pending = true;
  }
  if (spec.status === 'pending' || pending) {
    this.metrics.pendingSpecs++;
    this.display.pending(spec);
  } else if (spec.status === 'passed') {
    this.metrics.successfulSpecs++;
    this.display.successful(spec);
  } else if (spec.status === 'failed') {
    this.metrics.failedSpecs++;
    this.display.failed(spec);
  }
};

module.exports = PendingSpecReporter;

Specs marked as pending are displayed as pending in the report.

andregs · 2017-04-09T15:38:27Z

This hack works for me while we don't have angular/jasminewd#32

// In protractor `onPrepare`
jasmine.Suite.prototype.pend = function (message) {
  this.markedPending = true;
  this.children.forEach(spec => spec.pend(message));
};

xdescribe('Foo', function(){
  // bar
}).pend('reason');

Sankannavar · 2017-08-18T09:41:56Z

Am new to protractor
Was trying to skip some testcases on failure and continue remaining during batch run
I tried using pending but its not working inside it block
ex: it('Step 1:', function () {
});
it('Step 1:', function () {
if(true)
{
pending('my pending comment').it('my it block', function() { });
}
}); here am getting as failed
but same working outside of it block

it('Step 1:', function () {

});

pending('my pending comment').it('my it block', function() {  });

it('Step 2:', function () {
	
});

Please some one help me. thanks in advance.

Kitsula · 2018-06-08T18:30:32Z

Short workaround:
(environment.postDeployment ? xit : it)('should something', () => { expect(true).toBeTruthy(); });

aschlei · 2018-09-24T20:48:36Z

This issue should be fixed in jasmine 2.9 (See jasmine/jasmine@f4caf27). However, protractor is still using jasmine 2.8.

ViieeS · 2019-04-05T10:57:59Z

I have the latest version of Jasmine v3.3.0 and still get this behaviour... 😕

tymfear · 2019-04-05T11:12:13Z

I have the latest version of Jasmine v3.3.0 and still get this behaviour... 😕

If you don't use Protractor 6.x, then, whichever version of Jasmine you got installed, you will use the version used by jasminewd.

ViieeS · 2019-04-05T16:04:57Z

@tymfear oh, sorry don't test it with protractor. It's about unit tests.

nitin-kapoor · 2020-06-21T12:54:03Z

I am using pending function like below but its marking as failed
Even the spec report treating pending function as failed.

nitin-kapoor · 2020-06-21T12:55:55Z

I extended the jasmine-spec-reporter (https://www.npmjs.com/package/jasmine-spec-reporter)

var SpecReporter = require('jasmine-spec-reporter');
var PendingSpecReporter = SpecReporter;

PendingSpecReporter.prototype.specDone = function (spec) {
  this.metrics.stopSpec(spec);
  var pending = false;
  for(var i = 0; i < spec.failedExpectations.length; i++) {
    if ( spec.failedExpectations[i].message.toLowerCase().indexOf('pending') >= 0) pending = true;
  }
  if (spec.status === 'pending' || pending) {
    this.metrics.pendingSpecs++;
    this.display.pending(spec);
  } else if (spec.status === 'passed') {
    this.metrics.successfulSpecs++;
    this.display.successful(spec);
  } else if (spec.status === 'failed') {
    this.metrics.failedSpecs++;
    this.display.failed(spec);
  }
};

module.exports = PendingSpecReporter;

Specs marked as pending are displayed as pending in the report.

I have used this implementation and it's awesome!
But the issue is that even if we are using a pending function in the spec then jasmine is marking its status as failed only. I got to know while debugging(screenshot below) However I can add a little hack in this by adding checks for specific test cases by checking its description or id but this can cause issues later.

Can you please suggest something.

Test case shich is marked as pending

Garcia-JuanLuis · 2020-12-14T16:22:50Z

This issue should be fixed in jasmine 2.9 (See jasmine/jasmine@f4caf27). However, protractor is still using jasmine 2.8.

Hi, thank you for your support. I've updated to Protractor 7.0.0 and Jasmine 3.6.3.
When I use pending('Force skip') ==> I get an error that says "Failed: undefinedForce skip".
NOTE: I also see a message that says "no tests were found" which is pretty interesting.

Garcia-JuanLuis · 2020-12-14T20:21:07Z

I extended the jasmine-spec-reporter (https://www.npmjs.com/package/jasmine-spec-reporter)
var SpecReporter = require('jasmine-spec-reporter');
var PendingSpecReporter = SpecReporter;

PendingSpecReporter.prototype.specDone = function (spec) {
  this.metrics.stopSpec(spec);
  var pending = false;
  for(var i = 0; i < spec.failedExpectations.length; i++) {
    if ( spec.failedExpectations[i].message.toLowerCase().indexOf('pending') >= 0) pending = true;
  }
  if (spec.status === 'pending' || pending) {
    this.metrics.pendingSpecs++;
    this.display.pending(spec);
  } else if (spec.status === 'passed') {
    this.metrics.successfulSpecs++;
    this.display.successful(spec);
  } else if (spec.status === 'failed') {
    this.metrics.failedSpecs++;
    this.display.failed(spec);
  }
};

module.exports = PendingSpecReporter;
Specs marked as pending are displayed as pending in the report.
I have used this implementation and it's awesome!
But the issue is that even if we are using a pending function in the spec then jasmine is marking its status as failed only. I got to know while debugging(screenshot below) However I can add a little hack in this by adding checks for specific test cases by checking its description or id but this can cause issues later.

Can you please suggest something.

Test case shich is marked as pending

I extended the jasmine-spec-reporter (https://www.npmjs.com/package/jasmine-spec-reporter)

var SpecReporter = require('jasmine-spec-reporter');
var PendingSpecReporter = SpecReporter;

PendingSpecReporter.prototype.specDone = function (spec) {
  this.metrics.stopSpec(spec);
  var pending = false;
  for(var i = 0; i < spec.failedExpectations.length; i++) {
    if ( spec.failedExpectations[i].message.toLowerCase().indexOf('pending') >= 0) pending = true;
  }
  if (spec.status === 'pending' || pending) {
    this.metrics.pendingSpecs++;
    this.display.pending(spec);
  } else if (spec.status === 'passed') {
    this.metrics.successfulSpecs++;
    this.display.successful(spec);
  } else if (spec.status === 'failed') {
    this.metrics.failedSpecs++;
    this.display.failed(spec);
  }
};

module.exports = PendingSpecReporter;

Specs marked as pending are displayed as pending in the report.

This does not work for me; however, it could be the way I'm implementing it.
Would you place this code in its own class and then import it as a reporter?

I put the code under the Protractor config file, "onPrepare()" function... and I am getting two "results" in one window.

Failures:

Testing Skip Test shoud be skipped
Message:
Failed: => marked PendingSkipping This test
Stack:
Error: Failed: => marked PendingSkipping This test
at C:\regUSA\Utilities\ProtractorTest\node_modules\jasminewd2\index.js:64:48
at ControlFlow.emit (C:\regUSA\Utilities\ProtractorTest\node_modules\selenium-webdriver\lib\events.js:62:21)
at ControlFlow.shutdown_ (C:\regUSA\Utilities\ProtractorTest\node_modules\selenium-webdriver\lib\promise.js:2674:10)
at shutdownTask_.MicroTask (C:\regUSA\Utilities\ProtractorTest\node_modules\selenium-webdriver\lib\promise.js:2599:53)

1 spec, 1 failure
Finished in 0.012 seconds

               Pending                     *

Testing Skip Test shoud be skipped
No reason given

Executed 0 of 1 spec INCOMPLETE (1 PENDING) in 0.012 sec.
[15:17:09] I/local - Shutting down selenium standalone server.
[15:17:09] I/launcher - 0 instance(s) of WebDriver still running
[15:17:09] I/launcher - chrome #1 failed 1 test(s)
[15:17:09] I/launcher - overall: 1 failed spec(s)
[15:17:09] E/launcher - Process exited with error code 1

Garcia-JuanLuis · 2020-12-15T16:24:38Z

I took the implementation provided by sgibson21 and created a new project with Jasmine 3.6.3 and Protractor 7.0.0.
Still, I was getting failures when using pending(); so, I went ahead and started to play around with the implementation. Following is what my confi.js file looks like:
`// conf.js
let Jasmine2HtmlReporter = require('protractor-jasmine2-html-reporter');
var SpecReporter = require('jasmine-spec-reporter').SpecReporter;

let htmlReporter = new Jasmine2HtmlReporter({
savePath: 'reports',
fileName: 'e2e-report',
takeScreenshotsOnlyOnFailures: true,
cleanDestination: false,
fileNameDateSuffix: true
});

exports.config = {
framework: 'jasmine',
specs: ['spec.js'],

onPrepare() {
var PendingSpecReporter = SpecReporter;

PendingSpecReporter.prototype.specDone = function (spec) {
  this.metrics.stopSpec(spec);
  var pending = false;
  var pendingReason;
  for(var i = 0; i < spec.failedExpectations.length; i++) {
	  var index = spec.failedExpectations[i].message.toLowerCase().indexOf('pending');
	if ( index >= 0){
		pendingReason = spec.failedExpectations[i].message.substring(index+7); // 7 = the length of the word "pending"
		pending = true;
	}
  }
  if (spec.status === 'pending' || pending) {
	this.metrics.pendingSpecs++;
	this.display.pending(spec);
	
	spec['failedExpectations'] = [];
	spec['passedExpectations'] = [];
	spec['status'] = 'pending';
	
	if(spec['pendingReason'] === 'undefined' || spec['pendingReason'] === ''){
		spec['pendingReason'] = pendingReason;
	}
	
  } else if (spec.status === 'passed') {
	this.metrics.successfulSpecs++;
	this.display.successful(spec);
  } else if (spec.status === 'failed') {
	this.metrics.failedSpecs++;
	this.display.failed(spec);
  }
};

module.exports = PendingSpecReporter;

jasmine.getEnv().addReporter(new PendingSpecReporter({ spec: { displayStacktrace: 'pretty' } }));
    // Assign the test reporter to each running instance
jasmine.getEnv().addReporter(htmlReporter);

let jasmineReporter = require('jasmine-reporters');
jasmine.getEnv().addReporter(new jasmineReporter.JUnitXmlReporter({
  consolidateAll: true,
  savePath: 'reports',
  filePrefix: 'e2e-report',
}));

}
}`

Following is what my specs look like
`// spec.js
describe('Testing custom results', function() {
xit('shoud be skipped because we are using "x" in front of the spec', function() {
pending('Force skip');
expect(true).toBe(true);
});

it('shoud be skipped using custom implementation', function() {	 
  expect(true).toBe(true);
  pending('My custom pending reason!');

});

it('shoud pass!', function() {
expect(true).toBe(true);
});

it('shoud fail!', function() {
expect(true).toBe(false);
});

});`

Following is what the XML report looks like
`

-

-<testcase time="0.001" name="shoud be skipped because we are using "x" in front of the spec" classname="Testing custom results">

-

- (C:\regUSA\Utilities\ProtractorTest\spec.js:18:17) at C:\regUSA\Utilities\ProtractorTest\node_modules\jasminewd2\index.js:112:25 at new ManagedPromise (C:\regUSA\Utilities\ProtractorTest\node_modules\selenium-webdriver\lib\promise.js:1077:7) at ControlFlow.promise (C:\regUSA\Utilities\ProtractorTest\node_modules\selenium-webdriver\lib\promise.js:2505:12) at schedulerExecute (C:\regUSA\Utilities\ProtractorTest\node_modules\jasminewd2\index.js:95:18) at TaskQueue.execute_ (C:\regUSA\Utilities\ProtractorTest\node_modules\selenium-webdriver\lib\promise.js:3084:14) at TaskQueue.executeNext_ (C:\regUSA\Utilities\ProtractorTest\node_modules\selenium-webdriver\lib\promise.js:3067:27) at asyncRun (C:\regUSA\Utilities\ProtractorTest\node_modules\selenium-webdriver\lib\promise.js:2974:25) at C:\regUSA\Utilities\ProtractorTest\node_modules\selenium-webdriver\lib\promise.js:668:7]]>

`

Following is what the HTML report looks like
``

shadiabuhilal · 2021-05-27T21:02:04Z

To skip test block programmatically, please use done arg, and call done function before calling pending.

example:

it('promise pends', function(done) {
  return new Promise(function () {
    // Make sure to call done() before pending()
    done();
    pending('promise');
  });
});

juliemr added type: feature request framework: jasmine2 type: bug and removed type: feature request labels Sep 23, 2015

juliemr modified the milestones: Upcoming, 2.5 Sep 30, 2015

juliemr self-assigned this Oct 2, 2015

juliemr modified the milestones: Blocked, 2.5 Oct 2, 2015

bruderol mentioned this issue Apr 2, 2016

Extend Fluent DSL (former chained DSL) as a full featured Scenarioo DSL Example scenarioo/scenarioo-js#18

Closed

juliemr closed this as completed Jul 14, 2016

digitalkaoz mentioned this issue Apr 12, 2017

pending workaround for protractor bcaudan/jasmine-spec-reporter#135

Closed

RandomUser35453453234 mentioned this issue Oct 19, 2017

Disable taking screenshots while calling pending(); mlison/protractor-jasmine2-screenshot-reporter#115

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Calling `pending` in a spec marks the spec as failed. #2454

Calling `pending` in a spec marks the spec as failed. #2454

sveneh commented Aug 27, 2015

cstraynor commented Sep 8, 2015

theandrewlane commented Sep 14, 2015

lynchblue commented Sep 15, 2015

ctapobep commented Sep 20, 2015

juliemr commented Oct 2, 2015

bruderol commented Mar 18, 2016

bruderol commented Mar 18, 2016

akstatic commented Mar 30, 2016

lgarest commented Apr 13, 2016

moneytree-doug commented Apr 15, 2016

mattc41190 commented Apr 25, 2016

rcbop commented Apr 29, 2016

d-werner commented Apr 29, 2016

ArturKwiatkowski commented May 5, 2016

bruderol commented May 5, 2016

stheikki commented Jun 2, 2016

dverbovyi commented Jun 8, 2016 •

edited

Loading

HugoDoyon commented Jul 12, 2016

juliemr commented Jul 14, 2016

sgibson21 commented Feb 28, 2017

andregs commented Apr 9, 2017 •

edited

Loading

Sankannavar commented Aug 18, 2017

Kitsula commented Jun 8, 2018

aschlei commented Sep 24, 2018

ViieeS commented Apr 5, 2019

tymfear commented Apr 5, 2019

ViieeS commented Apr 5, 2019

nitin-kapoor commented Jun 21, 2020

nitin-kapoor commented Jun 21, 2020 •

edited

Loading

Garcia-JuanLuis commented Dec 14, 2020

Garcia-JuanLuis commented Dec 14, 2020

Garcia-JuanLuis commented Dec 15, 2020

shadiabuhilal commented May 27, 2021

Calling pending in a spec marks the spec as failed. #2454

Calling pending in a spec marks the spec as failed. #2454

Comments

sveneh commented Aug 27, 2015

cstraynor commented Sep 8, 2015

theandrewlane commented Sep 14, 2015

lynchblue commented Sep 15, 2015

ctapobep commented Sep 20, 2015

juliemr commented Oct 2, 2015

bruderol commented Mar 18, 2016

bruderol commented Mar 18, 2016

akstatic commented Mar 30, 2016

lgarest commented Apr 13, 2016

moneytree-doug commented Apr 15, 2016

mattc41190 commented Apr 25, 2016

rcbop commented Apr 29, 2016

d-werner commented Apr 29, 2016

ArturKwiatkowski commented May 5, 2016

bruderol commented May 5, 2016

stheikki commented Jun 2, 2016

dverbovyi commented Jun 8, 2016 • edited Loading

HugoDoyon commented Jul 12, 2016

juliemr commented Jul 14, 2016

sgibson21 commented Feb 28, 2017

andregs commented Apr 9, 2017 • edited Loading

Sankannavar commented Aug 18, 2017

Kitsula commented Jun 8, 2018

aschlei commented Sep 24, 2018

ViieeS commented Apr 5, 2019

tymfear commented Apr 5, 2019

ViieeS commented Apr 5, 2019

nitin-kapoor commented Jun 21, 2020

nitin-kapoor commented Jun 21, 2020 • edited Loading

Garcia-JuanLuis commented Dec 14, 2020

Garcia-JuanLuis commented Dec 14, 2020

Garcia-JuanLuis commented Dec 15, 2020

shadiabuhilal commented May 27, 2021

Calling `pending` in a spec marks the spec as failed. #2454

Calling `pending` in a spec marks the spec as failed. #2454

dverbovyi commented Jun 8, 2016 •

edited

Loading

andregs commented Apr 9, 2017 •

edited

Loading

nitin-kapoor commented Jun 21, 2020 •

edited

Loading