Since I simulate things, basically I loop running same kernels over and over. Large amount of those kernels don't change their argument values. For now I always set all the arguments before clEnqueueNDRangeKernel, over and over again.

For these unchangeable argument values, can I set arguments only at start ?

Because what bothers me is the "Notes", that are little ambiguous to me, from clSetKernelArg specs. With focus on this sentence:
Users may not rely on a kernel object to retain objects specified as argument values to the kernel.
Btw how high is clSetKernelArg's overhead anyway?