Testing BibleTime

This page has guidelines for testing.

Contents 1 General guidelines 1.1 Texts, layouts and sizes 2 Bookshelf index 3 Bookmarks 4 Bookshelf Manager 5 Display windows 5.1 Bibles 5.2 Commentaries 5.3 General books 5.4 Dictionaries 5.5 Other 6 Search 7 Configuration

General guidelines

Here we have collected some things which should help testers find out what to do. These are guidelines only. The problem with testing is that one user (let alone developer) can't find all ways an application can be used. Therefore it's important to play around and try to find some unexpected use cases. Some user may use the application daily in a way which never came to a developer's mind. Do what you usually or rarely do plus as many extra things as come to your mind. Be evil: try to think what the developers haven't considered and try to crash the application.

Testers should have a debug build if possible. Start with --debug option.

Install BibleTime using another user account. Create an account if necessary, preferably with a name with non-ascii characters. Do all tests.

Texts, layouts and sizes

Whatever you do, always think about the visible texts. Are they comprehensible and correct? Are they good English (if you aren't using a translated UI)? Are they informative? Do they mislead users? Are they consistent with each other, using same words and conventions?

Test the tooltips, too. Every component which is not self-evident should have a tooltip. See http://devel.bibletime.info/wiki/Programming_Guidelines#User-visible_text to know how they should have been written.

Change the general UI font (possibly with qtconfig or qtconfig-qt4 program, if changing the system wide fonts doesn't work). Try double the size and half the size of your usual font. Check through all the UI elements. You should be able to read all texts in one way or the other. If they aren't directly visible, resize the windows or areas. If some text can't be seen and the window or area can't be resized larger, it's a serious usability bug.

Every window should fit in the display. Some modern netbook computers have vertical resolution of 600 or even 480 pixels where desktop bar may take 40 or so pixels making the available space less than about 440 pixels. If a window can't be resized under that (with the normal font size), it's a usability bug.

Try to resize every window smaller and larger and check that they behave reasonably.

Bookshelf index

• Change Grouping and Hiding. Check that those are saved between sessions.
• Check that Grouping and Hiding are reflected in everywhere where expected (all places except Bookshelf Manager: Change/Add work buttons in text windows, search work list...).
• Use all RMB menu items for different works.
• Add and remove works and install folders with Bookshelf Manager (q.v.) and check that changes are reflected here.

Bookmarks

• Load existing bookmarks created with 1.6.
• Try different things with many or no bookmarks.
• Start drag from different places from BT or from other applications and drop to bookmarks.
• Drag to the bookmarks view to different areas (items, selections, empty area).
• Drag bookmarks to other widgets and other applications.
• Cancel drags (with ESC key) which are created in different places and are on top of different areas.
• Resize the bookmarks view, make it very small/large and try different things.
• Try dragging on top of a folder which has sub- and subsubfolders.
• Open the RMB menu on top of different areas with different kinds of item selections - does it have the correct items (dis)abled?
• Test the RMB menu items with different kinds of single items
• Test the RMB menu items with different kinds of multiple selections
• Copy and move different kinds of items and selections to different places. Do they really move/copy as expected?

Bookshelf Manager

Sources and install locations:

• Add sources, even broken ones.
• Delete sources.
• After adding and deleting, close the manager and open it again to see if the changes have been saved.
• After adding and deleting, close and open the whole app.
• Add install locations.
• Add locations with non-ascii (e.g. arabic) characters. Try to find and install works there.
• Remove install locations.
• After adding and deleting install locations the main bookshelf index and display windows should be updated automatically. (Add a location where works are already installed, or remove the same location, and see that module lists are updated.)
• Keep modules from all locations open in display windows while removing install locations.

Install modules:

• If possible, set up a private ftp server and a repository where you can download from.
• It's especially valuable to test with multicore or multiprocessor machines.
• Install one module.
• Install several modules in one go.
• Install several modules from several sources in one go.
• Cancel all installations with "stop/cancel all"...
  • immediately after the dialog has opened.
  • after downloading has started.
  • after all downloads have ended but the dialog is still visible.
• Cancel all installations with individual Stop/cancel button.
• Cancel some installations but let some continue.
  • Cancel while it's still waiting for turn.
  • Cancel when it's preparing but download hasn't started.
  • Cancel after download has started.
  • The first and last items are especially important.
• Try to install to non-writable destination location.
• Try to do anything else while installing: search, open and close modules, navigate in them, change configuration settings...
• After installing check that installed works are seen in the rest of the UI.
• After installing check that sources don't show those installed modules as still installable.

Remove modules:

• Remove all modules.
• Remove some modules.
• Remove one module.
• Keep modules open in display windows while removing modules.
• Try to do something else while removing (you have to be lightning fast:)

Display windows

Try each type of modules/texts. There are also some testing modules in ftp://labs.crosswire.org/pub/sword/experimentalraw/ repository, possibly having new features which are not yet supported by BT.

Bibles

• Navigate with each method (drag&drop a reference, write a reference, use dropdown lists, next/prev b/c/v buttons, history buttons, verse references within Bible text, crossreferences within Bible text). Use these methods one after another in different orders.
• Add, remove and change parallel texts.
• Change display settings.
• Use all RMB menu items.

Commentaries

Navigate just as with Bibles. Open the writable commentary and edit it.

General books

Navigate. Open a book you know and see if it has all chapters etc.

Dictionaries

• Test if dictionaries have all entries. Navigate.
• Test a dictionary which has non-ascii keys. E.g. LiddellScott.

Other

Test modules with images. E.g. ABSMaps from the Xiphos repo. Different image file formats may work differently, so try to find each of them.

Search

• Try at least one module from each module type/category.
• Try at least one module with non-ascii unicode characters.
• Try one Greek and one Hebrew module. (instead of or in addition to the previous item.)
• Try searching Strong's numbers (in KJV, some Greek modules).
• Try searching morphology (KJV, some Greek modules).
• Try searching adjacent Strong's numbers (some words in KJV represent more than one Greek word).
• Try searching for small words: articles, "and", "or"... alone and with other words.
• Try searching all available tags (also other than Strong's and morphology: heading, footnote).
• Try strange search terms: special characters etc.
• Try "All words", "Some words" and "Full" in combination with other options and search terms.
• Try selecting different sets of modules and searching in them.
• Selected modules have a history list - see that it works correctly.
• Try the scope/range.
• Edit and test scopes.
• Try the Strong's numbers feature in the module list in the results area.
• Try Analyzing the results and check if it looks correct.
• Open and close the search dialog in different situation while doing different things.
• Saving and restoring the dialog settings automatically (histories, user defined settings etc.)

Configuration

• Open and close Configuration dialog.
• Try every and each option. There's no need to list them here - there are too many and they may change. See that they work as expected.
• Close and open the application to see that settings have been saved and restored.
• If it's not restored, you can look from the config file if it's been saved at all, or just not restored.