Building programs with pre-compiled binaries

Hi,

I was wondering about the following: the OpenCL spec states that applications should try and build their CL sources once on the first run and then save the binaries created for use on subsequent runs. However, when considering how to load the binaries again, I wondered how I should save those binaries and load them onto the proper devices if the configuration is ever changed.

For example, suppose I have a platform with 2 devices, one of type CPU and one of type GPU. I load my sources, compile them and save the binaries to disk then continue to run my application.

Now the user modifies something on the system. This could range from replacing hardware (CPU, GPU) to installing another driver. What’s the proper way of dealing with this when my application runs again?

One way could be to just try and load all saved binaries into the driver and if that fails, just rebuild everything. Or is there some way to uniquely identify a device so I can be sure the binary will compile onto it?

Pre-compiled binaries are a bit tricky at the moment. There is no good way to identify a specific hardware device, and, even more importantly, a particular version of the OpenCL runtime with which it interacts. Without this sort of versioning, the best you can do is try to make sure that nothing has changed in your system or hope the runtime catches incompatibilities. This makes it extremely difficult to pass around binaries today. For example, if the Nvidia binaries are in PTX format, this will work across all their GPUs in theory. However, if they change something in the runtime/compiler with regards to their memory layout or libraries it may break without any good way to detect this.

You are correct, the correct way to look at binaries is as a quick-loading method that can fail at any time. Always keep a fallback path that recompiles your kernels from source when loading the binaries fails.

Detecting if a binary is invalid for a given device is very fast. You don’t need to worry about the performance cost of sending invalid/old binaries.

Hi,
I think trying to build and recompiling on error quite ugly.
I myself developed primitive configuration file solution. File is not meant to be edited manually - it stores info about builds. For ex.:
DEFINEPROGRAM P1
DEFINEBUILD P1 P1_frAQ.bld
CL_PLATFORM_NAME P1_frAQ.bld NVIDIA CUDA
CL_PLATFORM_VERSION P1_frAQ.bld OpenCL 1.0 CUDA 3.0.1
CL_PLATFORM_VENDOR P1_frAQ.bld NVIDIA Corporation
CL_DEVICE_TYPE P1_frAQ.bld CL_DEVICE_TYPE_GPU
CL_DEVICE_NAME P1_frAQ.bld GeForce GTX 275
CL_DEVICE_VENDOR P1_frAQ.bld NVIDIA Corporation
CL_DRIVER_VERSION P1_frAQ.bld 197.45
CL_DEVICE_VERSION P1_frAQ.bld OpenCL 1.0 CUDA

For each program the builds are defined and recognized by filename with .bld extension. I also store information about the platform/device the build was compiled on.
The application needs to be redistributed with opencl kernels sourcefiles if we want new builds to be compiled for platforms we haven’t provided binaries and entries in config file.

So it works like this:

  1. We pass program name.
  2. The platform and device information is being queried (clGet…Info functions)
  3. The config file is being searched for program build that matches current platform
    4a. If found it creates the program with binary stored in appropriate .bld file
    4b. If not found program is build with source (so in such situation the source is needed), then the binary is saved into .bld file and appropriate entries are added to configuration file.
    So on the next and following runs on the same host source code is not needed anymore since program is loaded with binary.

If we don’t expect the program to be run on configurations for which we haven’t prepared the builds we don’t need to include source files.
Simple and works. Sorry that opencl doesn’t include such tool forcing programmers to develop their own solutions.

If we don’t expect the program to be run on configurations for which we haven’t prepared the builds we don’t need to include source files.

In my opinion that is a bad idea. At any time, drivers may refuse to load old program binaries. Make sure that the applications you ship will load from source code if the binaries fail loading.

If you don’t do that, your application may fail the next time anything changes in the system, and customers don’t like that.

I don’t have customers :wink:
But yeah, generally you’re right. So actually if you want to have reliable app there’s no way yet to hide your kernels source.