Adding instance extensions in vulkan layer

Hi fellow vulkan programmers

I have a few questions about proper adding instance extensions and new layers to vkInstanceCreateInfo in vkCreateInstance inside another layer

  1. Firsly in layer manifest file there is a section with instance_extension

“instance_extensions”: [
{
“name”: “VK_KHR_get_physical_device_properties2”,
“spec_version”: “2”
}
]

Does it have only informative nature or does it actually do something ?
I didn’t notice any changes when i put that section in json file. I had
to add infotmation about that instance extension in vkCreateInstance either way.

  1. If my layer uses another layer if available should I put this information in json file (it is not meta layer) ?

  2. Assuming that vulkan application doesn’t fill vkInstanceCreateInfo with instance extensions and layers
    that I need. I destroy that instance and create another one with needed data added. But I’m not sure I did that
    correctly because on AMD graphic card it crashes while destroying old instance (it works on NVIDIA). It is possible
    to not destroy old instance and it will work but I don’t think it is proper way to do that.

VKAPI_ATTR VkResult VKAPI_CALL CreateInstance(
	const VkInstanceCreateInfo*                 pCreateInfo,
	const VkAllocationCallbacks*                pAllocator,
	VkInstance*                                 pInstance)
{
	VkLayerInstanceDispatchTable* pDispatchTable   = *((VkLayerInstanceDispatchTable **)*pInstance);
	VkLayerInstanceCreateInfo*    pLayerCreateInfo = (VkLayerInstanceCreateInfo *)pCreateInfo->pNext;

	// step through the chain of pNext until we get to the link info
	while (pLayerCreateInfo && (pLayerCreateInfo->sType != VK_STRUCTURE_TYPE_LOADER_INSTANCE_CREATE_INFO || pLayerCreateInfo->function != VK_LAYER_LINK_INFO))
	{
		pLayerCreateInfo = (VkLayerInstanceCreateInfo *)pLayerCreateInfo->pNext;
	}

	if (nullptr == pLayerCreateInfo)
	{
		// No loader instance create info
		return VK_ERROR_INITIALIZATION_FAILED;
	}

	PFN_vkGetInstanceProcAddr  vkNextGetInstanceProcAddr = pLayerCreateInfo->u.pLayerInfo->pfnNextGetInstanceProcAddr;
	// move chain on for next layer
	pLayerCreateInfo->u.pLayerInfo = pLayerCreateInfo->u.pLayerInfo->pNext;
	
	//........... Adding instance extensions and layers if available and not active code
	
	if(createNewInstance) // This variable is true if new extensions or layers has been added
	{
		VkLayerInstanceDispatchTable* pInstanceDispatch = *((VkLayerInstanceDispatchTable **)*pInstance);
		
		PFN_vkCreateInstance  pLoaderCreateInstance  = (PFN_vkCreateInstance)pInstanceDispatch->GetInstanceProcAddr(nullptr, "vkCreateInstance");
		
		PFN_vkCreateInstance pNextCreateInstance = (PFN_vkCreateInstance)vkNextGetInstanceProcAddr(VK_NULL_HANDLE, "vkCreateInstance");
		VkResult ret = pNextCreateInstance(&oldCreateInfo, pAllocator, pInstance); // Send information about creating instance to next layer

		PFN_vkDestroyInstance pLoaderDestroyInstance = (PFN_vkDestroyInstance)vkNextGetInstanceProcAddr(*pInstance, "vkDestroyInstance");

		pLoaderDestroyInstance(*pInstance, pAllocator); // Destroy instance with old vkInstanceCreateInfo. It crashes on AMD cards but works on NVIDIA

		VkResult result = pLoaderCreateInstance(&newCreateInfo, pAllocator, pInstance); // Creating instance with new vkInstanceCreateInfo it will call this function again
		return result;
		
	}
	
	PFN_vkCreateInstance vkNextCreateInstance = (PFN_vkCreateInstance)vkNextGetInstanceProcAddr(VK_NULL_HANDLE, "vkCreateInstance");

	VkResult ret = vkNextCreateInstance(pCreateInfo, pAllocator, pInstance);
	

	return ret;
}

I tested that code with VIA and it didn’t catch any errors. What did I do wrong ?

Not sure but it is mandatory (if layer provides extensions).
Documentation: Vulkan-Loader/LoaderAndLayerInterface.md at main · KhronosGroup/Vulkan-Loader · GitHub

If the official layers are any indication, you are supposed to expose vkEnumerateInstanceExtensionProperties, and return the extension property of the layer you are providing:

Ideally, layers should be self-contained. Otherwise you make the rules and behavior of your layer.
It might be nice to intercept the vkCreateInstance, and if the ppEnabledLayers does not contain the dependency layer, then return VK_ERROR_LAYER_NOT_PRESENT

You should not destroy the dispatchable handles. Other layers, and loader are using those.

Might be possible to hack it some way, but your code seems wrong. You are destroying instance, that was not even created yet.

Thanks

Actually I’m not destroying instance which was not created

if(createNewInstance) // This variable is true if new extensions or layers has been added
{
	VkLayerInstanceDispatchTable* pInstanceDispatch = *((VkLayerInstanceDispatchTable **)*pInstance);
 
	PFN_vkCreateInstance  pLoaderCreateInstance  = (PFN_vkCreateInstance)pInstanceDispatch->GetInstanceProcAddr(nullptr, "vkCreateInstance");
 
	PFN_vkCreateInstance pNextCreateInstance = (PFN_vkCreateInstance)vkNextGetInstanceProcAddr(VK_NULL_HANDLE, "vkCreateInstance");
	VkResult ret = pNextCreateInstance(&oldCreateInfo, pAllocator, pInstance); // Send information about creating instance to next layer
 
	PFN_vkDestroyInstance pLoaderDestroyInstance = (PFN_vkDestroyInstance)vkNextGetInstanceProcAddr(*pInstance, "vkDestroyInstance");
 
	pLoaderDestroyInstance(*pInstance, pAllocator); // Destroy instance with old vkInstanceCreateInfo. It crashes on AMD cards but works on NVIDIA
 
	VkResult result = pLoaderCreateInstance(&newCreateInfo, pAllocator, pInstance); // Creating instance with new vkInstanceCreateInfo it will call this function again
	return result;
 
}

I let the layer create unmodified instance and then I destroy that instance.
After that I create modified new one and all layers are notified about this fact because pointer to vkCreateInstance is acquired by vkGetInstanceProcAddress.
Problem is at

PFN_vkDestroyInstance pLoaderDestroyInstance = (PFN_vkDestroyInstance)vkNextGetInstanceProcAddr(*pInstance, "vkDestroyInstance");

I should notify all layers about destroying instance so that part should look like this

{
	VkLayerInstanceDispatchTable* pInstanceDispatch = *((VkLayerInstanceDispatchTable **)*pInstance);
	
	PFN_vkCreateInstance  pLoaderCreateInstance  = (PFN_vkCreateInstance)pInstanceDispatch->GetInstanceProcAddr(nullptr, "vkCreateInstance");
	
	PFN_vkCreateInstance pNextCreateInstance = (PFN_vkCreateInstance)vkNextGetInstanceProcAddr(VK_NULL_HANDLE, "vkCreateInstance");

	VkResult ret = pNextCreateInstance(&oldCreateInfo, pAllocator, pInstance);

	PFN_vkDestroyInstance pLoaderDestroyInstance = (PFN_vkDestroyInstance)pInstanceDispatch->GetInstanceProcAddr(*pInstance, "vkDestroyInstance");

	pLoaderDestroyInstance(*pInstance, pAllocator);

	VkResult result = pLoaderCreateInstance(&newCreateInfo, pAllocator, pInstance); // it will call this function again
	return result;
}

Unfortunately It won’t work on any GPU because call to pLoaderDestroyInstance will unload my layer DLL and after return from vkDestroyInstance
process will crash. But there is a way to avoid that by increasing reference count of loaded module

{
	VkLayerInstanceDispatchTable* pInstanceDispatch = *((VkLayerInstanceDispatchTable **)*pInstance);
	
	PFN_vkCreateInstance  pLoaderCreateInstance  = (PFN_vkCreateInstance)pInstanceDispatch->GetInstanceProcAddr(nullptr, "vkCreateInstance");
	
	PFN_vkCreateInstance pNextCreateInstance = (PFN_vkCreateInstance)vkNextGetInstanceProcAddr(VK_NULL_HANDLE, "vkCreateInstance");

	VkResult ret = pNextCreateInstance(&oldCreateInfo, pAllocator, pInstance);

	{
		//Increase reference count
		const uint32_t layerPathSize = 1024;
		wchar_t layerPath[layerPathSize];
		
		HMODULE layerHandle = GetModuleHandleW(L"LayerName.dll");
		GetModuleFileNameW(layerHandle, layerPath, layerPathSize);

		layerHandle = LoadLibraryW(layerPath);
	}

	PFN_vkDestroyInstance pLoaderDestroyInstance = (PFN_vkDestroyInstance)pInstanceDispatch->GetInstanceProcAddr(*pInstance, "vkDestroyInstance");

	pLoaderDestroyInstance(*pInstance, pAllocator);

	VkResult result = pLoaderCreateInstance(&newCreateInfo, pAllocator, pInstance); // it will call this function again
	return result;
}

In that case code above works.
I also looked on implementation of core validation layer and It looks like vkCreateInstane in core_validation.cpp doesn’t create modified instance to add new instance extensions.
To find out alternative way I would have to trace code from vkCreateInstance trampoline but if it works I’m not eager to do that. I have a feeling that VkInstanceCreateInfo
is modified even before vkCreateInstance of layer is called.

Thanks for your help