Great, I'm glad to hear you are aware of the issue. It's easy enough to work around by restarting the game entirely after a handful of game loads. I've also seen other unity games struggle with similar memory issues in the past, so hopefully it can be resolved here as well.
Thanks for the great game!