How to prioritize test cases and run using specific prioroty?

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

How to prioritize test cases and run using specific prioroty?

Bikash-3
Hi,

I am in need of prioritizing test cases such as p0, p1, p2, and p3. Then we would like to run p0 testcases whenever there's a new build and then only rest of the testcases if p0 passes fully. So, I was wondering if JUnit has such a feature out of the box. If not, could somebody point me how I can achieve this? Any help would be highly appreciated.

Reply | Threaded
Open this post in threaded view
|

Re: How to prioritize test cases and run using specific prioroty?

chet hendrickson
Hello Bikash,

It sounds like what you want is a test suite. http://www.clarkware.com/articles/JUnitPrimer.html#testsuite 

Thursday, June 9, 2011, 11:49:34 AM, you wrote:


 
Hi,

I am in need of prioritizing test cases such as p0, p1, p2, and p3. Then we would like to run p0 testcases whenever there's a new build and then only rest of the testcases if p0 passes fully. So, I was wondering if JUnit has such a feature out of the box. If not, could somebody point me how I can achieve this? Any help would be highly appreciated.






--
Best regards,
 Chet Hendrickson                          mailto:[hidden email]
 Check out our upcoming CSM Plus courses @
http://hendricksonxp.com/index.php?option=com_eventlist&Itemid=28

[Non-text portions of this message have been removed]

Reply | Threaded
Open this post in threaded view
|

Re: How to prioritize test cases and run using specific prioroty?

Cedric Beust
On Thu, Jun 9, 2011 at 9:59 AM, Chet Hendrickson <[hidden email]>wrote:

> Hello Bikash,
>
> It sounds like what you want is a test suite.
> http://www.clarkware.com/articles/JUnitPrimer.html#testsuite


Not really, it looks like he's asking for test
dependencies<http://testng.org/doc/documentation-main.html#dependent-methods>
.

--
Cédric


[Non-text portions of this message have been removed]

Reply | Threaded
Open this post in threaded view
|

Re: How to prioritize test cases and run using specific prioroty?

Bikash-3
Thanks for the replies. Our test harness is already implemented using JUnit. So, converting it to TestNG can be a hassle. If I want to keep JUnit but implement the Priorities, what could be the best way? It looks like I would need to override the runner method where we can check the priority and decide whether to execute or not. Any alternate way?

BikuDAA

--- In [hidden email], Cédric Beust â™" <cedric@...> wrote:

>
> On Thu, Jun 9, 2011 at 9:59 AM, Chet Hendrickson <lists@...>wrote:
>
> > Hello Bikash,
> >
> > It sounds like what you want is a test suite.
> > http://www.clarkware.com/articles/JUnitPrimer.html#testsuite
>
>
> Not really, it looks like he's asking for test
> dependencies<http://testng.org/doc/documentation-main.html#dependent-methods>
> .
>
> --
> Cédric
>
>
> [Non-text portions of this message have been removed]
>


Reply | Threaded
Open this post in threaded view
|

Re: How to prioritize test cases and run using specific prioroty?

Charlie Poole-7
In reply to this post by Cedric Beust
Priorities != Dependencies.

Charlie

2011/6/9 Cédric Beust ♔ <[hidden email]>

>
>
> On Thu, Jun 9, 2011 at 9:59 AM, Chet Hendrickson <[hidden email]
> >wrote:
>
>
> > Hello Bikash,
> >
> > It sounds like what you want is a test suite.
> > http://www.clarkware.com/articles/JUnitPrimer.html#testsuite
>
> Not really, it looks like he's asking for test
> dependencies<
> http://testng.org/doc/documentation-main.html#dependent-methods>
> .
>
> --
> Cédric
>
>
> [Non-text portions of this message have been removed]
>
>  
>


[Non-text portions of this message have been removed]

Reply | Threaded
Open this post in threaded view
|

Re: How to prioritize test cases and run using specific prioroty?

Cedric Beust
On Thu, Jun 9, 2011 at 12:22 PM, Charlie Poole <[hidden email]>wrote:

> Priorities != Dependencies.
>

I call priorities "soft dependencies" in TestNG: you want methods run in a
certain order but failures should not have any impact on what methods are
running.

In contrast, if b() has a hard dependency on a() and a() fails, then b()
won't be run and marked "skipped" in the report.

--
Cédric


[Non-text portions of this message have been removed]

Reply | Threaded
Open this post in threaded view
|

Re: How to prioritize test cases and run using specific prioroty?

Charlie Poole-7
FWIW, I only use the term "dependency" when b() actually depends on some
side
effect of a() in order to run successfully. Dependencies can only be
implemented
by running the tests in a certain order.

Calling something a "priority" means that it's more important than another
test, for
whatever reason reasons. Prioritization may be implemented by ordering the
tests
but don't have to be. For example, the high priority tests might be given
more
visibility in the report.

Note that I'm not talking about a particular implementation but about a
general
way of looking at dependency/priority/ordering. However, it is the logical
structure
I'm trying to use in my own work.

Ideally, I don't think we should tell test frameworks how to order tests,
but should
give indicators of true dependencies and priorities, letting the framework
decide
how to run the tests most appropriately.

Charlie


2011/6/9 Cédric Beust ♔ <[hidden email]>

>
>
> On Thu, Jun 9, 2011 at 12:22 PM, Charlie Poole <[hidden email]
> >wrote:
>
> > Priorities != Dependencies.
> >
>
> I call priorities "soft dependencies" in TestNG: you want methods run in a
> certain order but failures should not have any impact on what methods are
> running.
>
> In contrast, if b() has a hard dependency on a() and a() fails, then b()
> won't be run and marked "skipped" in the report.
>
>
> --
> Cédric
>
> [Non-text portions of this message have been removed]
>
>  
>


[Non-text portions of this message have been removed]

Reply | Threaded
Open this post in threaded view
|

Re: How to prioritize test cases and run using specific prioroty?

Cedric Beust
On Thu, Jun 9, 2011 at 12:40 PM, Charlie Poole <[hidden email]>wrote:

>
> Note that I'm not talking about a particular implementation but about a
> general
> way of looking at dependency/priority/ordering. However, it is the logical
> structure
> I'm trying to use in my own work.
>
> Ideally, I don't think we should tell test frameworks how to order tests,
> but should
> give indicators of true dependencies and priorities, letting the framework
> decide
> how to run the tests most appropriately.
>

Both options are useful, and TestNG also supports explicit priorities, so
you can specify the ordering either by:

   - Explicitly saying b() depends on a().
   - Saying that b() has a priority of 10, a() has a priority of 5 and let
   TestNG decide the ordering.

Overall, I find that depending on groups is more flexible: you don't say
that b() depends on a() but that it depends on the group "init", so that any
method that belongs to the group init will always be run before b(). It's
much more flexible when you want to add methods down the line.

--
Cédric


[Non-text portions of this message have been removed]

Reply | Threaded
Open this post in threaded view
|

Re: How to prioritize test cases and run using specific prioroty?

Dale Emery
In reply to this post by Bikash-3
Hi Bikash,

I am in need of prioritizing test cases such as p0, p1, p2, and p3. Then we
> would like to run p0 testcases whenever there's a new build and then only
> rest of the testcases if p0 passes fully. So, I was wondering if JUnit has
> such a feature out of the box. If not, could somebody point me how I can
> achieve this? Any help would be highly appreciated.
>
Do you mean that if the system fails even one p1 test, you don't want to
know whether it would pass /any/ of the p2 and p3 tests?

Dale

--
Dale Emery
Consultant to software teams and leaders
Web: http://dhemery.com


[Non-text portions of this message have been removed]

Reply | Threaded
Open this post in threaded view
|

Re: How to prioritize test cases and run using specific prioroty?

David Saff
In reply to this post by Bikash-3
Bikash,

How are you running the tests?  For example, if you're using ant,
there's some promising directions.

   David

On Thu, Jun 9, 2011 at 11:49 AM, Bikash <[hidden email]> wrote:

> Hi,
>
> I am in need of prioritizing test cases such as p0, p1, p2, and p3. Then we would like to run p0 testcases whenever there's a new build and then only rest of the testcases if p0 passes fully. So, I was wondering if JUnit has such a feature out of the box. If not, could somebody point me how I can achieve this? Any help would be highly appreciated.
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: How to prioritize test cases and run using specific prioroty?

Gerard Meszaros-2
Could't you create a Category for each priority and then use an ant script that
runs each Category in turn as long as all the tests pass?

These two links might help get you started:

http://stackoverflow.com/questions/2176570/how-to-run-all-tests-belonging-to-a-certain-category-in-junit-4
http://stackoverflow.com/questions/6226026/how-to-run-all-junit-tests-in-a-category-suite-with-ant

Gerard

On 6/10/2011 7:37 AM, David Saff wrote:

>
> Bikash,
>
> How are you running the tests? For example, if you're using ant,
> there's some promising directions.
>
> David
>
> On Thu, Jun 9, 2011 at 11:49 AM, Bikash <[hidden email]
> <mailto:abjaisi%40live.com>> wrote:
> > Hi,
> >
> > I am in need of prioritizing test cases such as p0, p1, p2, and p3. Then we
> would like to run p0 testcases whenever there's a new build and then only rest
> of the testcases if p0 passes fully. So, I was wondering if JUnit has such a
> feature out of the box. If not, could somebody point me how I can achieve
> this? Any help would be highly appreciated.
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
>

--
Gerard Meszaros
Lean/Agile Coach/Mentor/Trainer
http://www.gerardmeszaros.com
1-403-827-2967

Author of the Jolt Productivity Award winning book "xUnit Test Patterns - Refactoring Test Code" and winner of the "Programming with the Stars" competition at Agile 2009. Learn more at http://xunitpatterns.com/index.html






------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/junit/join
    (Yahoo! ID required)

<*> To change settings via email:
    [hidden email]
    [hidden email]

<*> To unsubscribe from this group, send an email to:
    [hidden email]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

Reply | Threaded
Open this post in threaded view
|

Re: How to prioritize test cases and run using specific prioroty?

sudha raman
In reply to this post by Bikash-3
best to use is TestNG many features are available according to your need google it.

best hands on in selenium software testing course in Chennai