[0] https://www.linuxatemyram.com/

[1] https://news.ycombinator.com/item?id=26897095

[2] https://github.com/Feh/nocache

https://github.com/lima-vm/lima/blob/master/examples/default... 4gig default

~$ free -m total used free shared buff/cache available Mem: 3907 304 1446 1 2156 3431 Swap: 0

Thank you.

It looks like Fedora CoreOS inside podman maybe doesn't do this?

    [root@localhost ~]# free -h
                   total        used        free      shared  buff/cache   available
    Mem:           7.8Gi       212Mi       7.2Gi       8.0Mi       333Mi       7.3Gi
    Swap:             0B          0B          0B

Docker based workflows are rubbish on my MBP.

I think Linux running flawlessly, with full hardware acceleration, all the drivers and equal or close battery life on a MBP would make it the best laptop money could buy. Until that day, it's a ferrari with square wheels

Not sure what you mean by that, hypervisor is what allows the VMs to exist.

Anyway, containers are a Linux thing so I use a Linux laptop when working with containers. No VM is needed and life is great.

Hypervisors are one method of running virtual machines, you also have classical virtualisation methods and when I bring up the virtual machines being slow, I always get someone who pushes up their glasses with an "well actually its a Hypervisor running it, so it's essentially native speed".

I find Linux is a much nicer development environment all around though.

[Edit] Yup, already got one.

What do you mean by “classical virtualisation methods”, actual CPU emulation so you can run e.g. PowerPC on x86?

There is nothing unclear about what I said, I understand that hypervisors are supposed to be fast "almost native speed!". I also know that running Docker/Podman in Linux on MacOS isn't fast for many reasons.

I don't think misnome is trying to be pedantic either, I think we're both unclear as to what you're trying to say.

Edit: I'm also dubious that hypervisors always offer "native" speed. To me this seems like it depends a lot on the hypervisor, the workload, the guest OS, etc.

But anyway prior to that it used VirtualBox (on Mac) if I remember correctly, which is a full virtualisation environment.

Anyway, taking a guess at what you're saying: do you mean full software virtualization vs. hardware-assisted virtualization? If so, I would agree with you that even best-case performance of HA VMs is not quite bare-metal; sometimes it's close enough to not matter much.

Virtualised OS are obviously "slightly" slower because they now don't have 100% of a system to use, and usually require a dedicated chunk of the system RAM, but usually what people actually mean by saying they are slower is because often they don't have direct hardware peripheral access (e.g. their own dedicated network card and disk drives) and the hypervisor emulates some amount of the system. This is the case with the most accessible linux VMs that run docker, because simulating the block devices on top of the MacOS filesystem is slow.

Docker itself is none of these, it's a fancy chroot enabled by linux kernel features, so _requires_ running on top of linux or a sufficiently linux-like compatibility layer.

So you see why it's somewhat incoherent and makes it sound like you are talking out of your ass when you complain that Hyperkit is a hypervisor but "Virtual machines aren't", but all these horrible people keep disagreeing with you.

Currently trialing nix-shell (https://nixos.org/manual/nix/stable/command-ref/nix-shell.ht...) in the iOS team (this team doesn't need to deploy software to servers, so its a small trial for now to make sure certain build tools are useable and on the right version for our build), but see no reason why it couldn't extend to other similar workflows in other teams.

My problem with Nix is, just like Rust, while it provides a very high ROI, the barrier to entry seems very steep; and unlike Rust, the documentation still leaves a lot to wish for. I'd go all-in on it right now, but I'm (rightfully, I guess?) scared of Nix-unique problems that will take 10x the effort to address than the mess that we're currently dealing with.

Tried switching to podman with docker-compose and podman-compose, but can’t get the auth to work for private repos on dockerhub. So basically just accepting my fate of crazy idle memory and cpu usage.

No root cause after 3 years. The best guess is the overhead of IO somewhere in the stack.

- a PHP-fpm container running a 5m LoC codebase - a nodeJS container running a webpack dev server instance on a 1m LoC frontend codebase - 2 elasticsearch containers (one for logs and 1 for for app search) - 2 Kibana container - 1 logstash container - A rabbitmq container - a memcached container - a redis container - a Percona container - 3-4 micro services each in there own container. - an imgproxy container

All of this runs quite snappily in ~6 GB of RAM and I’ve never noticed any slowness while also running Edge, slack, VSCode, PHPStorm, zoom, and a bunch of terminal windows, and I could probably get that down a bit by consolidating the elasticsearch and kibana instances.

It wasn't the only factor though. The lack of options for laptops and inability to self repair drove me away too. Used to be one of the strong points.

The root of the problem is that VM memory is allocated on demand, but never freed after it's used for the first time. In other words, once used, it can never be released. Since my app has a lighter userspace, it starts out using less memory than other VMs, but eventually reaches the memory limit given enough usage and time. (My optimized memory management setup means the VM works well with a lower memory limit than others, but it doesn't solve the fundamental problem.)

Linux has a feature called "page reporting" to report chunks of memory that are no longer used to the hypervisor, which can then drop the to reduce usage on the host side. WSL 2 actually uses this feature, but I suspect it becomes less effective with longer VM uptime because memory becomes more fragmented over time. Since Hyper-V has been limited to dropping contiguous 2 MiB chunks of memory until recently [1], fragmentation is likely the reason many users report high memory usage. Page cache is definitely a contributor as well, but a much easier one to fix. It looks like Microsoft is working on the problem with page reporting.

Although Apple's Virtualization.framework doesn't support page reporting, I was able to implement it with some workarounds and confirmed that this works with QEMU on Linux. Unfortunately, while free memory is correctly reported to macOS, nothing actually seems to get freed. I'm planning to report this to Apple because it seems like memory ballooning (essentially a more limited and primitive version of page reporting) doesn't work as documented, whether it's Virtualization.framework or another VMM like QEMU. If/when Apple fixes this, it'll be possible to reduce memory usage significantly. Details from my investigation into what's going on with memory management on the XNU side: https://twitter.com/kdrag0n/status/1612309883411640321

The good news: From my testing, the issue isn't as bad as it appears. The "free" memory tends to compress quite well, so XNU's memory compression does a good job at taking care of it when you're actually running low on memory.

(Shameless plug on this topic: The app I'm working on already has quite a few improvements over others: fast networking (30 Gbps), VirtioFS and bidirectional filesystem sharing, Rosetta for fast x86, full Linux (not only Docker), lower CPU usage, native Swift UI, and other tweaks. Email in bio for waitlist. Details to avoid spamming this thread: https://news.ycombinator.com/item?id=34374176)

[1] https://lkml.org/lkml/2022/9/30/81

> This feature is powered by a Linux kernel patch that allows small contiguous blocks of memory to be returned to the host machine when they are no longer needed in the Linux guest. We updated the Linux kernel in WSL2 to include this patch, and modified Hyper-V to support this page reporting feature. In order to return as much memory to the host as possible, we periodically compact memory to ensure free memory is available in contiguous blocks. This only runs when your CPU is idle. You can see when this happens by looking for the ‘WSL2: Performing memory compaction’ message inside of the output of the dmesg command.