Output buffers
Part of an ongoing series of essays tentatively entitled Don’t embarrass me, Don’t embarrass yourself: Notes on thinking in C and Unix.
Taylor and Jamie have been having some trouble when they assign to string
variables. Like many sensible programmers, they’ve been trying to isolate
the problem by writing simple programs in which they can replicate the
error. Like many less-than-sensible programmers, they are using printf
statements to keep track of what’s happening in their program.
Consider the following program, which they’ve written to see where in memory a pointer is stored and whether or not they can assign to it.
/**
* expt1
* A simple experiment with pointers and dereferences.
*
* <insert FOSS license>
*/
// +---------+-------------------------------------------------------
// | Headers |
// +---------+
#include <stdio.h>
#include <malloc.h>
// +-----------+-----------------------------------------------------
// | Constants |
// +-----------+
// The longest string length
#define MAXLEN 32
// +------+----------------------------------------------------------
// | Main |
// +------+
int
main (int argc, char *argv[])
{
char *ptr = malloc (MAXLEN);
printf ("Assigning to element 0 of %p ... ", ptr);
*ptr = 'a';
printf ("done\n");
return 0;
} // main
What do you expect this program to do?
- A: Crash, with no output other than the error message?
- B: Crash, printing something in addition to the error message?
- C: Print
Assigning to element 0 of 0x??????? ... done
, where the question marks represent some address? - D: None of the above?
- E: All of the above?
Let’s check
$ ./expt1
Assigning to element 0 of 0x1d69010 ... done
$ ./expt1
Assigning to element 0 of 0x1fa5010 ... done
$ ./expt1
Assigning to element 0 of 0x187f010 ... done
It looks like the correct answer was C.
Jamie and Taylor note that they often have trouble when they don’t initialize pointers, and make a slight change to the program above see what happens.
int
main (int argc, char *argv[])
{
char *ptr;
printf ("Assigning to element 0 of %p ... ", ptr);
*ptr = 'a';
printf ("done\n");
return 0;
} // main
What do you expect this updated version to do?
- A: Crash, with no output other than the error message?
- B: Crash, printing something in addition to the error message?
- C: Print
Assigning to element 0 of 0x??????? ... done
, where the question marks represent some address? - D: None of the above?
- E: All of the above?
Let’s check.
$ ./expt2
Segmentation fault
It appears that A is the correct answer. Why do you think that happened?
Think about it.
Think some more.
Here’s the most typical answer: It crashes in the
printf
statement, since
ptr
refers to an unknown area of memory.
Do you agree? Why or why not?
Their classmates, River and Sam, note that they find it easier to follow what’s happening if they print each message on a separate line. Here’s their version of the same program.
int
main (int argc, char *argv[])
{
char *ptr;
printf ("Assigning to element 0 of %p.\n", ptr);
*ptr = 'a';
printf ("Done assigning.\n");
return 0;
} // main
What do you expect this updated version to do?
- A: Crash, with no output other than the segfault message?
- B: Crash, printing something in addition to segfault message?
- C: Print
Assigning to element 0 of 0x???????.
on one line, andDone assigning.
on a separate line, where the question marks represent some address? - D: None of the above?
- E: All of the above?
Earlier, you said that the printf
was causing it to crash [1]. So that
means you think the answer should still be A, right? Let’s check.
$ ./expt3
Assigning to element 0 of (nil).
Segmentation fault
Interesting. This time, we got some output. What’s the difference, other than the newline (and some slightly different text)? Nothing! So why did we get output in this version, and not the previous version?
The answer is fairly simple and straightforward [2].
The standard implementation of printf
(and some variants thereof) buffers
output. That is, there’s a large array of characters reserved for the
output. When you call printf
, instead of printing immediately,
printf
stores the output into the buffer until it’s necessary to print
that output.
Why have a buffer? It turns out that there’s comparatively high expense in generating output from a program, and that the expense is not that much different for a few characters and a lot of characters. Our programs become more efficient if the program sends the output only when necessary.
What are the necessary
conditions for printf
to actually generate the
output? There are four possibilities [3]: The buffer might fill; printf
might include a newline character; the client could explicitly indicate
that they want the buffer sent to output [4]; or the program could terminate
without crashing. Oh, I guess there’s something like a fifth possibility:
You can turn off buffering. I’ll let you figure out how on your own.
For the second version of the program, none of those conditions held, so we saw no output. For the third version of the program, there was a newline, so the output appeared. Let’s try explicitly flushing the buffer.
int
main (int argc, char *argv[])
{
char *ptr;
printf ("Assigning to element 0 of %p ... ", ptr);
fflush (stdout);
*ptr = 'a';
printf ("done\n");
return 0;
} // main
Let’s see what happens.
$ ./expt4
Assigning to element 0 of (nil) ... Segmentation fault
Of course, no sensible C programmer prints log messages to stdout
.
Log messages should go to stderr
or to a file. Let’s look at yet
another variant in which we use stderr
. Note that we’ve dropped
the newline from the first fprintf
.
int
main (int argc, char *argv[])
{
char *ptr;
fprintf (stderr, "Assigning to element 0 of %p ... ", ptr);
*ptr = 'a';
fprintf (stderr, "done\n");
return 0;
} // main
What do you expect this updated version to do?
- A: Crash, with no output other than the segfault message?
- B: Crash, printing something in addition to segfault message?
- C: Print
Assigning to element 0 of 0x??????? ... done.
on one line, where the question marks represent some address? - D: None of the above?
- E: All of the above?
Since this program looks very close to version 2, which printed nothing, you
might assume that the answer should once again be A. But you’d be wrong.
It turns out that stderr
, by default, is an autoflush
buffer. That is,
anything you print to stderr
gets printed immediately. Let’s see.
$ ./expt5
Assigning to element 0 of (nil) ... Segmentation fault
What should you take from this discussion?
- Output in C is more complex than you thought, with
stdout
(and, perhaps, other streams) relying on associated buffers.
- Amazingly, not knowing this issue can lead you to make incorrect assumptions about where your code is segfaulting.
- If you are using
printf
to debug, you should really usefprintf(stderr, ...)
. - Of course, you shouldn’t use either to debug. That’s why debuggers exist [5].
- If you are one of the people who was frustrated that input had to be on a separate line from the prompt because all prompts seemed to need newlines, now you know how to print the prompt so that the input can be entered on the same line [7].
Exercises
Determine how big the output buffer is. You might try reading documentation. You might try writing a program. You might try some combination thereof.
Figure out how to make
stdout
autoflush.
[1] Actually, I have no idea what you said. I just know that most students
think it’s the call to printf
.
[2] Well, I think the reason is simple and straightforward. Your mileage may vary.
[3] At least there are four possibilities of which I am aware.
[4] This last operation is called flushing
the buffer, which I think
says a lot about the designers’ impressions of most program output.
[5] We will consider gdb
, the GNU Debugger, at some point. I’ve forgotten
the name of the awesome debugger I used while in grad school (early 1990’s).
All I remember is that you could download a theme song [6] where the
developer sings and plays guitar, with a refrain of It’s the best damn
debugger that there’s ever been.
I wonder if I’ll ever figure out what it
was.
[6] In aiff
format, if I remember correctly.
[7] Something like the following.
printf ("Please enter something: ");
fflush (stdout);
fscanf ("%d", &val); // [<a href="#referent8" id="reference8">8</a>]
[8] Note that using fscanf
to read integers is potentially dangerous.
Version 1.0.1 of 2017-01-27.