Logging in Qt land

Sooner or later you’ll want to add logging to your Qt program. What are your options? After looking around the web a few years ago, I was disappointed to find out that there was no official Qt logging library. There is some support in Qt for logging, but you won’t get a logger component and it’s not immediately obvious how to log into a file.

The third party support that I had found wasn’t particularly encouraging:

  • There was a Log4Qt based on Log4j that was at version 0.1 back then. It has since reached version 0.3 at  the beginning of 2009, but there are no check-ins for the last 12 months.
  • Then there was the QxtLogger class, part of the qxt library. This one looked much more promising, but the API is a bit cumbersome (you have to pass the things that you want to log as QVariants  or as a QVariantList) and it was LGPL, which meant an extra dynamic library dependency.
  • A multitude of non-Qt C++ loggers, heavyweights like log4cxx or Pantheios.

What I really wanted was a simple logging library that knew about Qt types that I could just drop into my project in a few cpp files and then forget about it. After reading the Qt docs, I wrote a proof of concept logger with multiple destinations and logging levels. It would use qInstallMsgHandler to set up a static function that would build a message based on the text received from qDebug(), the current date and time,  the log level and send it to a list of destinations.

That was the same implementation that I cleaned up and decided to publish yesterday, only to find out in the mean time that using qInstallMsgHandler is not a good idea at all…

The simple solution that fails

The Qt docs mention that one could install a custom message handler that is called by the qDebug function family when doing the output. Basically, when you write qWarning() << “Parameter out of range:” << 1.0 << << “-” << 2.0, the custom message handler will see an array of char “Parameter out of range: 1.0 – 2.0″ and the message type, which is one of debug, warning, critical and fatal.

There are two issues with this approach:

  • the custom message handler must be a free function
  • there are too few logging levels!

My original implementation easily solved the first problem by using a singleton logger class. The custom handler just called Logger::instance() and was itself a static member of Logger, which meant that it had access to all its member variables.

The second problem was more tricky, but I managed to get around it by writing a special character at the beginning of the log message depending on the log level. If I wanted to log an info message, I would write QLOG_INFO() << “message”, which was translated into qDebug() << QsLogging::detail::infoMessage << “message”. The custom handler would then remove the QsLogging::detail::infoMessage from the message and pass it on to each destination.

Simple and it worked: I could use more logging levels and log to multiple destinations with a convenient syntax. Unfortunately, this custom handler had some subtle bugs.

For starters, when you install a custom message handler, all messages from qDebug/qWarning/etc will be sent to your handler. This means that Qt’s warnings or asserts would also end up in the handler and that the logger itself could trigger another log call. This could result either in a deadlock or an endless loop.

I had two choices: not use any Qt types in the custom handler and the log class or detect Qt’s messages and send them to the default handler. I chose the second, only to find out that there is no default handler, or at least not always.

qInstallMsgHandler can return NULL as the old handler. The function that calls the custom message handler is named qt_message_output and lives in qdebug.h. The docs mention that

The default message handler prints the message to the standard output under X11 or to the debugger under Windows. If it is a fatal message, the application aborts immediately.

The defaults are coded directly in qt_message_output, which checks to see if there’s any handler installed. My old handler was zero so I couldn’t forward Qt’s messages to it and I couldn’t forward them to my log. Even if I could somehow access the default handler, it was not thread safe, which meant that I would have to use platform specific synchronization (QMutex would not be available).

Custom message handlers were starting to be more trouble than they were worth, so I started looking at the qDebug implementation.  qDebug just returns an instance of the QDebug object, which does all the message formatting and sends the final message to qt_message_output when it’s destroyed.

The even simpler solution

Maybe I could use QDebug to get all the formatting for free? The QDebug constructor can take a QtMsgType debug level, a QIODevice or a QString.  Only when a QtMsgType is used will the QDebug destructor call qt_message_output.

As an example you could easily write a class MyFileLog which keeps  a QFile and gives you access to a QDebug(MyFileLog::theFile) for logging. In my case, I used the QString constructor and a wrapper to make QDebug give me the formatted log message and notify the logger that it should send the message to the destinations.

The final implementation is available at my bitbucket repository under a BSD license. It features a file and debugger destination, 6 logging levels, level filtering and can be added directly into a project. Since it’s based on QDebug, you can log most Qt types such as QVector or QHash directly or extend it for other types.

My conclusion is that you can use qInstallMsgHandler to control Qt’s messages, but you should use something more flexible for logging. Give QsLog a try and let me know what you think. Bugs & suggestions can be submitted directly at the repository.

ScopeGuard for signals: first try

While working with state machines in Qt I have encountered the typical situation where a state can be exited through a “fail” transition or a “success” transition. In my case, both transitions were triggered by signals and the signals were emitted from a checker function.

The success transition was easy since there’s usually only one way to do things right, but the failure transition could be triggered from multiple places inside the checker function. Every few lines I would check for some condition, do a few things, emit failed(), and return. Every few other lines I would forget to emit failed() and get some interesting bugs. Fun as that was, I got tired of it pretty fast… hence SignalGuard!

SignalGuard is a ScopeGuard that works with the moc and signals instead of template tricks and arbitrary functions. It automatically emits a signal when the SignalGuard object is destroyed, simplifying emits when dealing with multiple exit points.

Here’s a quick example:

SignalGuard cancelGuard(this, SIGNAL(requestCanceled()));
// do check one, if failed show a message, return
// do check two, if failed show a message, return
// do check three, if failed show a message, return
cancelGuard.dismiss(); // phew, we've made it

The signal guard will automatically emit requestCanceled when it is destroyed unless we dismiss() it. A maintenance programmer that will add other checks won’t introduce a subtle bug by forgetting to emit the canceled signal and the code is cleaner too!

Here’s another example:

emit uiWait(); // show a busy cursor
foo();
bar();
emit uiFinishWait(); // show a regular cursor

Without SignalGuard, we’d probably have to rewrite the above code or risk being stuck with an hourglass cursor:

emit uiWait();
try
{
  foo();
  bar(); // what if this throws?
}
catch( std::exception& )
{
  emit uiFinishWait();
  throw;
}
emit uiFinishWait();

But why complicate the control flow when you could write this instead:

SignalGuard g(this, SIGNAL(uiFinishWait()));
emit uiWait();
foo();
bar();

That’s about all that SignalGuard does right now. You can download the sources from my brand new bitbucket repository. The code is really simple – that’s why this is labeled first try – but I’d like to extend it to allow specifying the connection type (easy) and signals/slots with parameters (tricky, I need the parameter types).

QTL or STL?

First, a little history: when I started working with Qt I was coming from a STL/Boost background and was accustomed to using the STL containers and strings. On the first Qt project that I’ve worked on, a Linux port for a large-scale C++ system, the team’s approach was to use Qt strictly for the UI and the STL, Boost and other libraries for everything else.

At that time, I didn’t even know there was a QTL and I was using std::(w)string pretty much everywhere. After working on more Qt projects, I now use the QTL everywhere and have extensive experience with both libraries. My conclusion is that there is no clear winner in this comparison, that the choice can be quite complicated depending on the project type, and switching from one library to the other poses some subtle difficulties.

Containers

The STL and Qt have a different philosophy when it comes to containers:

STL

  • The containers are very parameterizable and built for maximum speed.
  • Has performance guarantees for each container.
  • Each implementation can use copy-on-write as it sees fit.
  • Free, generic functions are prefered to member functions.
  • There are multiple container implementations, just as there are multiple standard library implementations.
  • … and those implementations are not binary compatible, but they must obey the C++ standard. This includes the interfaces and performance guarantees.
  • Each implementation is free to do whatever optimizations that it pleases as long as it respects the standard.
  • Provides regular iterators.
  • Only includes a hash in TR1.
  • Only includes a variable length array in TR1.
  • Have a header-only implementation and the compiler has access to all the source code.
  • Has the <algorithm> header.

QTL

  • Containers are not parameterizable. Don’t like the default memory allocator? Well, that’s too bad…
  • Provides a list of performance guarantees in the docs.
  • Use copy-on-write to avoid needless expensive copies.
  • Lots of member functions provided for convenience. e.g: contains, indexOf, removeAt, takeAt, uniqueKeys, startsWith, etc
  • One implementation that is binary compatible between major releases but not major versions. i.e: Qt 4.5 is compatible up to Qt 4.9, but Qt 3.2.2 is not compatible with Qt 4.0
  • Qt containers do optimizations which are available on all operating systems.
  • Can use STL-style iterators or Java-style iterators.
  • Includes a hash and multihash implementation.
  • Includes QVarLengthArray.
  • Are not header only. Some functions are not available when building your program and they can’t be used for global optimizations.
  • Has the <QtAlgorithm> header.

If you will use the Qt containers together with the Qt framework, you will get a few other advantages:

  • Qt containers are streamable by default with QDataStream. This means that you only have to write a streaming operator for your user defined contained type.
  • They are used extensively in Qt’s API. If you use STL containers you will almost certainly have to make conversions.
  • Some signal slots connections copy their parameters. Since the Qt containers use COW, copying them is less costly.
  • You can use Q_FOREACH with Qt containers. Since Q_FOREACH does a copy of the target container, it would be too expensive to use with the STL.

My opinion is that the biggest advantage of the QTL is that it has the same implementation (including binary compatibility) on all OSes supported by Qt.  Some STL implementations might be below par when it comes to performance or they might be missing functionality. Some platforms don’t even have an STL! On the other hand, the STL is more customizable and is available in its entirety in header files… Like I said, there is no clear winner.

Strings

When deciding about using QString instead of std::(w)string, there are few points to consider:

  • QString scores a lot higher than std::basic_string when it comes to usability.
  • It offers Unicode support.
  • It lives in a dll/so, which means that whole program optimizations won’t be available. This can be worked around by linking with Qt statically, but the static vs dynamic debate is quite complicated and a decision has wider reaching effects.
  • Doesn’t allow customization through character traits or custom allocators. In fact, QString is a regular class, not a template and it can easily be forward declared.

Personally, I almost always use QString if I’m using Qt. I originally used conversion helpers to avoid a deep Qt dependency, but unless most of your program is Qt-free, it’s not worth it to use both QString and std::string.

Not a drop in replacement

Qt containers are STL-compatibile in the way that they provide iterators that match the STL algorithm library and the container interfaces mimic the STL ones. In addition to this, there’s an STL compatibility mode that allows you to directly convert to and from a STL container when you’re working with its Qt counterpart. e.g: QVector::toStdVector

Even so, you can’t swap between Qt and STL containers as easily as replacing a type definition:

  • QVector::at doesn’t do range checking besides an assert.
  • QList provides index-based access and is more similar to a std::deque. QLinkedList is probably the closes thing to std::list.
  • There are no constBegin, constEnd, constFind, etc functions in the STL. You might not easily be able to take advantage of Qt’s COW implementation.
  • The QMap and QMultiMap interfaces differ from the std::map and std::multimap ones. In particular, there’s no equal_range function.
  • Qt containers can’t be parametrized with an allocator.

Smart pointers

Thiago Maceira from Nokia/ex-Trolltech has done an excellent summary of the Qt smart pointers. If you read this summary, you’ll know 99% of what’s to know about Qt’s smart pointers.

I have added only a few observations from experience:

  • Unfortunately TR1 is not as wide spread as I would like at this point and I find it much easier to use the Qt shared pointer instead of introducing a large Boost dependency in a project.
  • QScopedPointer supports d-pointers and custom deleters. AFAIK, neither scoped_ptr nor auto_ptr do.

Avoid early pessimizations with Qt

Containers

First of all, know your data structures. Then, make sure you understand what kinds of optimizations the Qt containers employ under the hood. Ready? Let’s get to business then!

As you’ve probably read in the Qt Quarterly article, Qt containers (and other expensive to copy classes) use an optimization known as copy-on-write. The basic idea is that when a container is copied, the new copy shares the original container’s data instead of recreating it. Only when the data is modified does the modified container detach from the shared data and incurs the full copying cost. Assuming that you can’t avoid making copies of containers, the next best thing is to make sure that you don’t detach unless you have to.

Here are the tips:

  • Use Q_DECLARE_TYPEINFO to inform the containers whether your type can be moved in memory.
  • Use at instead of the operator[] if you need to access a container element but don’t need to change it. The former returns a const reference to the element, while the latter detaches before returning a reference to the element.
  • Use constBegin and constEnd. As an alternative, call begin/end on a const_cast<T>(container), as the const versions don’t detach.
  • Use constFind when working with hashes or maps. You’ve guessed it: find detaches.
  • Set a breakpoint inside the detach function and watch for unexpected calls.
  • Use reserve() when you know the size of the container and the container supports it.
  • Always use const T& for user data types with foreach/Q_FOREACH

Strings

Here are my recommendations for using Qt strings:

  • Do use char* when working with string literals, not everything has to be a QString. Qt understands UTF-8 and it can create QStrings from it.
  • Use QLatin1String when you have defined QT_NO_CAST_FROM_ASCII.
  • QString uses the same copy-on-write implementation as the containers. Pay attention when calling various member functions that could detach. e.g: use at() instead of operator[], constBegin() instead of begin(), etc
  • QStringRef provides fast access to substrings without creating new QString objects.
  • Qt 4.6: #include QStringBuilder to get fast concatenation with the % operator. This method avoids multiple memory allocations.

Finally, here are some performance recommendations straight from the Qt devs:

  1. Copy on write performance
  2. STL vs QTL
  3. Qt DevDays 2009 performance talk
  4. Fast container iteration
  5. Graphics performance overview
  6. Raster graphics performance
  7. OpenVG performance

Qt hidden features

Ok, maybe not hidden hidden, but still, these are not talked about so often: 

  • Q_GADGET offers some meta-object features without the need to inherit QObject. Combined with Q_ENUMS it allows reflection for enums.
  •  You can declare a single signal or slot by using the Q_SIGNAL and Q_SLOT macros.
  •  Q_FOREACH and Q_FOREVER!
  •  qChecksum offers a CRC implementation.
  •  qCompress / qUncompress.
  •  qDeleteAll deletes each element in a container.
  •  qRound does float to integer rounding.
  •  qFuzzyCompare: safe compare for float and double values.
  •  qVersion: Get the *runtime* qt lib version as a string value.
  •  QSysInfo header: word size, endianness, OS version.
  •  QtEndian header: endian conversions.

QtCreator tips and tricks

Here are a few tips and tricks for working with QtCreator:

Project management

  1. QtCreator understands pro files and it can use them to build your project, but under the hood it just uses qmake, so you’ll have to learn the qmake syntax to use QtCreator to its full potential. The qmake help files are good, and the syntax is ridiculously simple compared to make or automake.
  2. If you’re using the Visual Studio integration, the easiest way to convert your solution to a pro file is to select the “Create basic .pro file” menu entry and all the pro files will be generated for you. This doesn’t always work perfectly, but it gives you a good start: pro files are text files and they can easily be edited by hand. In my experience, the automatic converter can’t keep pro <-> sln projects in sync, you will have to tweak them and keep them in sync manually for best results.
  3. When you’re building something a bit more complicated than your average GUI program, you will probably organize your project into multiple libraries and one executable. One way to do this is by using the SUBDIRS template, but you will have to edit the pro files by hand. The other way is creating a new session in Qt Creator, opening each library + the binary as a separate project and configuring their dependencies from the Projects mode.
  4. Speaking of libraries, qmake won’t recompile a binary if one of its library dependencies changes: you will have to use the qmake PRE_TARGETDEPS variable for that. Unfortunately, the complete library name must be used, -llib won’t work.
  5. You can use QtCreator with the Visual C++ compiler without installing Visual Studio: download the Windows SDK for your OS, the Windows Debugging tools and the Qt MSVC binary package from Nokia and you’re set.

Debugging

  1. If you can’t debug your programs (program stops with signal received after starting debug), you might check if you have the COMODO firewall installed. If you do and you want to debug with QtCreator you’ll have to uninstall COMODO and reboot. Note: seen with QtC 1.2.1
  2. If you can’t get the debugger dumpers (aka visualizers) to work, do the following:
    • make sure that the debug dumpers are enabled in options
    • try recompiling the visualizers from the Qt versions dialog
    • try right clicking on the debug pane and select reload visualizers.
    • if all else fails, check the debugger view tab from the Debug mode for clues (show it by going to the Debug menu and clicking Views)
  3. If the MinGW debugger dumpers compilation fails, try removing all traces of MSVC from the LIB, INCLUDE and PATH environment variables and retry to compile it.
  4. If you’re using MinGW, right click the debugger variables view and check “Dereference pointers automatically” to get rid of the extra tree level between the value of a variable and its name.

User interface

  1. Click Esc to close the bottom panes and/or the help pane. Click Esc again to return to the editor from any other view.
  2. Press F1 to show side-by-side help. Press F1 again to move to the help view.
  3. The locator supports wildcard characters like * and ? Not as good as VA X’s multi word filter, but it’s a welcome feature!

That’s about it for now. I’ve seen some cool stuff in the pipeline on gitorious and I’m looking forward to an even better version of Qt Creator!