How to remotely capture a core dump on a Mac ?


Mac OS X is usually a very stable OS, over the last 10 years I can count on my fingers the rare number of cases when the system entirely crashed and required a reboot.

(I am not counting the deliberate attempts to make it crash , playing with development version of kernel modules or very very early builds of all kinds)

Since a couple of months however, I am able to crash the entire system just using iTunes.

When downloading multiple files simultaneously or when starting any type of concurrent disk access for several minutes, iTunes freezes, bringing the whole system down with him.

It looks like this is linked to my type of machine (iMac 24″) as I can not reproduce on any other three machines at home.  I am not the only one experiencing this issue.  Apple’s support forums host a thread here and another here with people describing exact similar symptoms (some of them with PowerMacs too)

As I am part of the Apple developer Program, I opened a bug report to request attention about this.

Surprisingly, Apple answered with 2-3 weeks asking for more details, in particular, a stack trace of the system while in frozen state.

Stackshot can be enabled as follows:

sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.stackshot.plist

Then, type the following chord:

Control-Option-Command-Shift-Period

a file will be written out /Library/Logs/stackshot.log and a symbolized file will be written to /Library/Logs/stackshot-syms.log

Unfortunately, this does not help in my case as the whole system was unresponsive when the crash occurs, I have no possibility to type any command while the system is frozen.

Apple then asked me to produce a core dump, using Non Maskable Interrupt trigger and a remote core dump server.

Basically, the idea is to configure the kernel to let it send a core dump to a remote server, through the network.  On another machine, you’ll have to setup the core dump server, i.e. a process that will listen on the network, will accept the connection from the crashed kernel and will dump the data to a file on disk.

Once the kernel client and the core dump server configured, you need to tell the kernel to generate the data and send them to the remote server.  For this top happen, you’ll need to force a panic with DTrace for example.

But in my case, the kernel is not panicking, just freezing.

I had to generate a Non Maskable Interrupt to activate the debugging process.  Once properly configured as described in Apple Technical Note, this is a simple as shortly pressing the POWER button.

This method is explained with many details and options in Apple technical Note TN2118.  It is surprisingly easy to setup and allows to collect valuable debugging information for developers.  I am attaching a screen shot of the crashed machine while in this state.

I hope Apple will be able to provide a fix for this issue.

,

  1. No comments yet.
(will not be published)