Open source C++ execution trace framework

At froglogic, we’re big fans of open source software. A large part of our engineering (and management!) staff contributed or contributes to open source projects, and everyone visiting our offices for a job interview certainly gets a big +1 in case she can show off some open source work! We also use a lot of open source software for our daily work, ranging from obvious projects like Git or the Linux kernel to individual libraries serving very specific purposes; the Acknowledgements Chapter of the Squish manual gives an impression of how tall the giants are upon whose shoulders we’re standing.

Over the last couple of years we contributed back various bug fixes and improvements to different projects we’re using, but we’d like to step things up a little bit. Hence, we now open-sourced an internally developed C++ framework called ‘TraceTool’ and made it available under the LGPL v3 license on our GitHub account:

http://github.com/froglogic/tracetool

TraceTool started out as a project which we did a few years ago but since grew to a full-fledged, efficient and configurable framework which we use ourselves internally in order to aid debugging our software.

TraceTool GUI highlighting trace statements generated by interesting function.

TraceTool GUI highlighting trace statements generated by interesting function.


(more…)

Squish tip of the week: Automating Multiple Applications with Multiple Squish Installations or Editions

Automating Multiple Applications with Multiple Squish Installations or Editions

Did you know that it is possible to use multiple Squish editions in a single test script?

The following example describes the setup and workflow for such scenario utilizing Squish for Qt and Squish for Web.

  1. Install Squish for Qt.
  2. Install Squish for Web.
  3. Create a Squish for Qt test suite with the Squish for Qt IDE.
  4. Create a Squish for Web test suite with the Squish for Web IDE.
  5. (more…)

Squish tip of the week: Automation Setup on Windows

Automation Setup on Windows
There are situations in which test scripts will hang and time out on Windows while executing. Most of this cases involve the lack of drawing/painting the application on the “screen”.

The underlying problem is that if the GUI of the application cannot be rendered, its objects do not have coordinates on the screen and testing software, like Squish, cannot find out where to perform a mouse click.

“The AUT did not respond to network communication” would be one indication of a test script execution problem in the error log.
(more…)

QtCon: Squish for Qt Training in Berlin

QtCon

On September 1st, as part of the QtCon conference, our partner KDAB hosts a day of training. This training day allows you to gain knowledge in several Qt-related topics, including automated Qt GUI testing with Squish for Qt.

froglogic‘s ISTQB-certified Senior Software Trainer Florian Turck will conduct the full-day Squish for Qt training and share his in-depth experience of effectively using Squish. Seats for the training are still available.

Register for QtCon here:

https://conf.qtcon.org/en/session/new?conference_acronym=qtcon

Bug Location with Squish Coco 3.4

Locating a software failure can be a tenuous and a time intensive operation due to the fact of the explosion of the source code size and their test suite. Also the complexity of some software imposes an organization in which the test team is separated from the development teams. In this condition, it is generally difficult to find with a simple test report the code parts responsible for a bug. A common way to facilitate the correction is to provide additional information like code dumps, execution traces, screen shots…
Code coverage metrics can also be provided. This metrics consists of recording if a source code item was executed or not. This is less precise that execution traces, but it has the advantage that this information is always available for the project on which it is also used to measure the quality of the code.
One way to use this information, is to compare a failed with a passed test. As explained in a previous post, the difference in the executed code can give a hint to the location of the bug. But on huge project with a lot of tests it is difficult to choose the pertinent tests and there is no guarantee that the failed test is executing more code as the passed tests.

A better approach is to process all coverage data globally with an algorithm which scans the execution of all passed and failed tests. I present here the philosophy of it as implemented in Squish Coco v3.4.
(more…)

Squish tip of the week: Automated Qt and QML GUI testing with Squish (Webinar)

View a full recording of the webinar* about automated Qt and QML GUI testing with Squish. During the Webinar, several questions were answered which can be found below the video.

View more Webinars and Tutorials in our Video Library.

(more…)

Visual Testing of elements and controls with Visual Verification Points

Agile Testing with Visual Verfication Points

What is Visual Testing?

Visual testing is a testing activity that aims to verify the presentation properties of a GUI object under various conditions. Testing of GUI based systems and software consists of the validation of both the GUI visual layout and its functionality.

Visual testing provides developers with the ability to monitor what was happening at the point of software failure by presenting the data in such a way that the developer can efficiently find this information.

For example, checking that a given object has the correct font size can be handled in isolation, while verifying that two objects are appropriately aligned will require comparing the position properties of both of them.
(more…)

Improved Management Of Object Names – Part 2

In a previous blog entry, we introduced an alternative, script-based, approach to maintaining a mapping of symbolic names (variables) to the actual object names (strings). By using script variables instead of free-form strings, the mapping was no longer stored in a separate text file but instead it was brought into the domain of the programming language, thus enabling a lot of tooling support e.g. for renaming symbolic names or finding references of a symbolic name in the test code.

However, there are still some annoyances related to managing object names like this, most notably caused by the fact that the actual object names are still plain strings:

Object Names As Strings Cause Trouble

  1. Composing object names is awkward and error-prone. For instance, assume that there are two object names for an imaginary Add User dialog and an OK button in the objectsmap.js file we introduced in the previous blog article:
    addUserDialog    = "{type='Dialog' text='Add User'}"
    okButton         = "{type='Button' text='OK'}"
    

    Since there can be multiple OK button objects visible at the same time, you might want to introduce a dedicated object name which specifically references the OK button in the Add User dialog by using the special container constraint. You might do:

    addUserOKButton  = "{container={" + addUserDialog + "} type='Button' text='OK'}"
    

    …i.e. use plain string concatenation. However – notice the mistake we made? The addUserDialog variable has a value which is already enclosed in curly braces, so we don’t need to (in fact: must not) specify them again when using the variable. So what we actually should have done is:

    addUserOKButton  = "{container=" + addUserDialog + " type='Button' text='OK'}"
    
  2. Implementing parametrised object names is fragile. To implement ‘parametrised symbolic names’ (which a programming language usually calls ‘functions’ or ‘procedures’ or ‘subroutines’) you could define a function such as
    def dialogName(title):
        return "{type='Dialog' text='" + title + "'}"
    

    This works well for many invocations such as dialogName(“Add User”) to generate the object name {type=’Dialog’ text=’Add User’}, but there’s a bug in here: what if the title contains a single quote character, e.g. for a dialog which is shown when a file could not be written to disk as in:

    cannotWriteFileDialog = dialogName("Can't Write File");
    // OOPS: cannotWriteFileDialog is set to {type='Dialog' text='Can't Write File'}
    

    This generates an invalid object name, causing an error at runtime.

  3. Both of these issues are only detected so late (when actually using the object name) because the object names are plain strings. Had we used some more expressive data structure, standard tools such as syntax checkers of ‘lint’-style programs might have detected the issue before we even ran the test.
    (more…)