Quality Center and BPT testing experiences

At my current customer we are working extensively with Business Process Testing (BPT) in Quality Center 10, and now recently ALM 11.52.

As you may know, BPT is an implementation of keyword-driven test automation.  The idea of BPT (or keyword-driven testing) is very nice:

  • test analyst can define Business Components (BC’s) (or keywords)
  • test engineer can implement these BC’s
  • test analyst can use the BC’s in test cases

BPTThe benefits can be big:

  • Nice separation between test analyst and test engineer.  It can be difficult to find a test analyst who can also write test code, or a test engineer who is willing to learn the business processes.
  • Test ‘scripts’ become more readable in keyword form.

HP has several documents and articles on BPT:

Unfortunately, we have two issues: 1) BPT in Quality Center is badly implemented, and 2) our usage of BPT is also badly implemented.

1) BPT in Quality Center is badly implemented

  • The editor doesn’t allow copy/past within a BPT test. This has been a problem in QC10, it still is a problem in ALM11.52 and apparently it is now possible to open a BPT test in UFT.  Guess what: it is not possible to open 2 BPT testcases in one UFT session, therefore it still is not possible to copy several steps between BPT testcases.  It is also not possible to copy a few BC calls several times in a test case; the BC has to be imported every time.  Very frustrating!
  • The editor isn’t flexible.  In QC10 the editor displayed the BC’s in a grid like manner: each row is a call to a BC, the columns were the names of the BC’s, status, input parameters, output parameters, …
    BPT_QC10As you can see, if BC’s have several parameters, they are all listed. This means only a few BC’s fit on the screen.  The user has lost the overview of the test case.
    Retrieving all the values of the parameters made this view also very slow.
    In ALM11.52 the editor changed: in grid view:
    BPT_ALM11_gridYes, it loads faster now and the test has the overview of the complete test case. However, what HP giveth, HP taketh away also: parameters are now not as easily visible: an extra screen has to be opened.  And you know HP: never remember the settings for a window! So every time you have to resize the window and columns.
  • The editor doesn’t allow to comment BPT test or flow steps. This makes it difficult to read the test case when editing it, but also the test report becomes less readable. For the last effect, we have made a “Add_comment” BC that gives info in the test report.  However in ALM11.52 the parameter is not visible in the BPT editor. Other solution is to group a few steps and you can rename the group to a more meaningfull name.  Grouping adds another extra click to retrieve the parameters of the BC’s in the group.  And the group name doesn’t come back in the test report…
  • BPT has a disastrous effect on our QLM performance.  In QC10 we had to turn of version control because we have so many BPT tests.  A few months ago we updated to ALM11.52 with version control turned on.  After a week, we had to turn off version control again due to performance reasons.

There are several more issues with BPT/ALM, but these are the most important ones for us.

2) Our usage of BPT is badly implemented

  • We have too detailed BC’s.  Our BC’s should be on the level of “CreateDepartureFlight with these parameters”.  However, they are on the level of “Goto_Tab”, “Open FlightEditor”, “Edit Flight”, “Save flight” with several steps in between.  This has several consequences:
    • Test cases become unreadable because there is too much detail in them. Since ALM11.52 doesn’t show the parameters anymore it is difficult to see what exactly the test case does.
    • Test execution becomes very slow. In QC10 this was even worse, in ALM11.52 HP introduced the BPTWrapperTest.  This speeds up the test execution (or at least the starting of the test). Sometimes you might want to disable it.
    • BC code becomes less maintainable, since each BC is implemented in a separate .vbs file. Now we are moving to an architecture where multiple BC’s are implemented in one .vbs file and the BC files only make calls to those files.

All these issues compounded lead to an almost unworkable, instable test automation ‘solution’.  Time to search for something else…