What is MergeKit? 

MergeKit is an open-source library designed for the fusion of pre-trained language models - a technique known as Model Merging. The library was created by Charles Goddard, an award-winning software engineer known for his pioneering work at NASA (and now the Chief of Frontier Research at Arcee AI). MergeKit has quickly gained popularity among LLM (Large Language Model) developers. It utilizes an out-of-core approach, enabling complex model merges even in environments with limited resources. MergeKit can perform merges using only the CPU or, if preferred, can leverage GPU acceleration with as little as 8 GB of VRAM. This toolkit facilitates efficient and flexible model merging, making it a valuable resource for developers working with pre-trained language models.