Get Specifics with Golang Memory Profiler

I created a web-server using Golang that ran machine learning algorithms for a video game. I was aware that the algorithms will take a large number of resources as the player data increases but I saw very large and inconsistent memory spikes that could use up to 4 to 6 gigs of RAM before going back to normal. Working with a web server that only had an 8GB of RAM in total and shared it with multiple services, I couldn’t have it so I attempted to investigate.

Golang has pretty good documentation for its profiler. I would recommend going through it before doing anything else since it shows the complete ability of the profiler. I worked with it to produce the following code.

https://gist.github.com/RaheelYawar/431ea1751da4caa1ff9171467cbd9d03

When your Golang application runs the

executeMemoryProfiler()

function, it will export the memory profile in the “mem.prof” file. The next step is to render this to show a memory trace. The documentation and many examples point you towards the following command:

go tool pprof mem.prof

You can then type “web” in the terminal to render your memory profile using a web browser. The problem is that you don’t see a lot of information which you can work with. I found a helpful blog post that addressed this issue. Try again but use the following command:

go tool pprof --alloc_space mem.prof

Again type “web” to render the profile. Now, you should see a more detailed flowchart of execution and the memory allocated to each function in that chart.

I hope this helps out someone, there isn’t a whole lot of information available about the quirks of memory profiling so I thought it would be a good idea to have a very basic post about it. Thanks for reading.

Originally published at http://raheelyawar.com on October 3, 2018.