No. Available RAM size does not limit your file-storage. While RAM is usually limited, the total size of the file-storage is not limited by MicroStream, but depends only on your hardware.
It is called in-memory, because MicroStream itself runs within the JVM process of your app and queries as well as CRUD operations are executed on the object graph in the RAM. The file-storage contains only the persistent data, just like a kind of backup. The persistent data are shared to many single files. By using channels you can even share the file-storage to different locations. Every channel has it's own file-storage. Thus, the max. size of a file-storage should be really very huge.
Lazy-loading works similar to JPA, which means you can load single reference on demand. To use lazy-loading you only have to define your classes lazy. Only when the object is read by get-method, the data are loaded by MicroStream automatically behind the scenes. If the object is in the memory already, you get the data directly from the RAM.
There are several different ways to save memory. Lazy objects can be removed from RAM. Removed objects are automatically loaded again when needed. Collections should should be always lazy.
In addition to that you should compare different JVMs. I have read that the Eclipse OpenJ9 JVM is very good with memory management and provides up to 30% lower memory footprint than Oracle's HotSpot JVM.