{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Capturing C-level stdout/stderr with `wurlitzer`\n",
"\n",
"Sometimes in Python you are calling some C code.\n",
"Sometimes that C code makes calls to `printf`,\n",
"or otherwise writes to the stdout/stderr of the process."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import ctypes\n",
"libc = ctypes.CDLL(None)\n",
"\n",
"try:\n",
" c_stderr_p = ctypes.c_void_p.in_dll(libc, 'stderr')\n",
"except ValueError:\n",
" # libc.stdout is has a funny name on OS X\n",
" c_stderr_p = ctypes.c_void_p.in_dll(libc, '__stderrp')\n",
"\n",
"\n",
"def printf(msg):\n",
" \"\"\"Call C printf\"\"\"\n",
" libc.printf((msg + '\\n').encode('utf8'))\n",
"\n",
"def printf_err(msg):\n",
" \"\"\"Cal C fprintf on stderr\"\"\"\n",
" libc.fprintf(c_stderr_p, (msg + '\\n').encode('utf8'))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"IPython forwards the Python-level `sys.stdout` and `sys.stderr`,\n",
"but it leaves the process-level file descriptors that C code will write to untouched.\n",
"That means that in a context like this notebook, these functions will print to the terminal, because they are not captured:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"printf(\"Hello?\")\n",
"printf_err(\"Stderr? Anybody?\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With wurlitzer, we can capture these C-level functions:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from wurlitzer import pipes, sys_pipes, STDOUT, PIPE"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"with pipes() as (stdout, stderr):\n",
" printf(\"Hello, stdout!\")\n",
" printf_err(\"Hello, stderr!\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"and redisplay them if we like:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Hello, stdout!\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Hello, stderr!\n"
]
}
],
"source": [
"import sys\n",
"sys.stdout.write(stdout.read())\n",
"sys.stderr.write(stderr.read())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Some tools, such as the IPython kernel for Jupyter,\n",
"capture the Python-level `sys.stdout` and `sys.stderr` and forward them somewhere.\n",
"In the case of Jupyter, this is over a network socket, so that it ends up in the browser.\n",
"\n",
"If we know that's going on, we can easily hook up the C outputs to the Python-forwarded ones with a single call:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Hello from C, 0!\n",
"Hello from C, 1!\n",
"Hello from C, 2!\n",
"Hello from C, 3!\n",
"Hello from C, 4!\n"
]
}
],
"source": [
"import time\n",
"\n",
"with sys_pipes():\n",
" for i in range(5):\n",
" time.sleep(1)\n",
" printf(\"Hello from C, %i!\" % i)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can also capture the pipes to any writeable streams, such as a `StringIO` object:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Hello, stdout!\n",
"Hello, stderr!\n",
"\n"
]
}
],
"source": [
"import io\n",
"\n",
"stdout = io.StringIO()\n",
"with pipes(stdout=stdout, stderr=STDOUT):\n",
" printf(\"Hello, stdout!\")\n",
" printf_err(\"Hello, stderr!\")\n",
"\n",
"print(stdout.getvalue())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## IPython extension\n",
"\n",
"You can also enable wurlitzer as an IPython extension,\n",
"so that it always forwards C-level output during execution:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"%load_ext wurlitzer"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Hello from C, 0!\n",
"Hello from C, 1!\n",
"Hello from C, 2!\n",
"Hello from C, 3!\n",
"Hello from C, 4!\n"
]
}
],
"source": [
"for i in range(5):\n",
" time.sleep(1)\n",
" printf(\"Hello from C, %i!\" % i)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.1"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
Generated by dwww version 1.15 on Sun Jun 16 21:00:25 CEST 2024.