Keeping bash history in sync on disk and between multiple terminals

I rely on my ~/.bash_history extensively to avoid retyping things, both short-term and long term (when I want to look up something I typed yesterday or last week).

First, an obvious necessity is

shopt -s histappend

in ~/.bashrc, so that bash just keeps on adding to the end of the ~/.bash_history file rather than obliterate the file at regular intervals.

By default, bash reads from your ~/.bash_history once when you log in to a terminal, and it updates ~/.bash_history once when you log out. I dislike both of these default behaviors.

Writing out to ~/.bash_history only when you log out is no good, because what if your terminal dies unexpectedly? You'll lose the history of any commands you typed there. This can happen if you kill a bash process, or if the machine powers off unexpectedly, or who knows what. So, to manually update your ~/.bash_history immediatety, at any time you can use this:

history -a

which tells bash to append to ~/.bash_history any commands in the current terminal that aren't already in there.

The second behavior of bash I dislike is that it reads from the history file only at login, mostly because I always have many terminals open at once. I run a command from one terminal, then run a different command from another, switch back and forth etc. These terminals may be appending all kinds of nice new lines to the end of my ~/.bash_history, but because bash never reads the history file except at login, they're never accessible except in the terminal where the command was originally run. Luckily you can force bash to re-read your history file via:

history -n

which tells bash to read any new lines that have appeared in ~/.bash_history since the last time it looked.

Now, say I type a really long command, e.g. a huge long Perl one-liner, in one terminal, and I want to then access and run that command in another terminal via bash history. With bash's default behavior, I can't do it; each terminal has its own history. What I could do is type history -a in the first terminal, and history -n in the second terminal; then it'd work wonderfully.

I find the thought of having to do that all the time tedious, though. Better would be for bash to do it for me every time I run a command. You can accomplish this by putting something like this in ~/.bashrc:

export PROMPT_COMMAND="history -a; history -n"

PROMPT_COMMAND lets you specify a command that bash will run every time it shows you a fresh command prompt, i.e. every time you run a command and the command finishes. So the above tells bash to read any new lines that have appeared in ~/.bash_history since the last time it read it, and then append the last-run command from this terminal to ~/.bash_history, every time you run a command.

So now, if you type a command in one terminal, and want to access it via the history of another terminal, run a command in the other terminal (or just hit Enter) to trigger PROMPT_COMMAND, and then your history will be nicely up-to-date and synchronized with any other terminals you have open. Almost certainly, you'll never notice the tiny bit of overhead caused by bash constantly reading and writing to ~/.bash_history.

See man bash for more info on the history builtin.

September 14, 2007 @ 7:55 AM PDT
Cateogory: Linux

30 Comments

bed
Quoth bed on November 11, 2007 @ 8:56 PM PST

Thank you!

shopt -s histappend was exactly what I searched for.

I had found this moths ago at a German blog but could not find it again... Today the "last logout wins" bites me again :(

Now, this should be history :)

Brian
Quoth Brian on November 12, 2007 @ 12:02 PM PST

Pun intended?

Anonymous
Quoth Anonymous on November 22, 2007 @ 11:32 AM PST

Your use of PROMPT_COMMAND works nicely, but it has one downside: Ctrl-O becomes much less useful. Normally, if you enter a series of commands, you can go back to the first one, and hit Ctrl-O repeatedly to run the current command and retrieve the next. If each terminal saves each command right after running it, then you can't necessarily reconstruct the series of commands run on one terminal.

Brian
Quoth Brian on November 22, 2007 @ 11:39 AM PST

Yeah, another bad thing is when for example you open a terminal just to start some server. Then you open another terminal and play around for a while. Then you want to restart the server process, so you go back to that terminal and Ctrl+C it and hit UP to go back in your history to re-run the server command. But oops, now a lot of commands are in between you and the one you want.

Maybe it'd be nice to script a way to easily choose which behavior you want on a terminal-by-terminal basis.

Much Ado About Nothing » Blog Archive » Sick of losing bash history?

[...] this post by Brian today, I will be doing this on all my [...]

Mike Golvach
Quoth Mike Golvach on May 18, 2008 @ 12:57 PM PDT

Nice post,

That's one of those things I've just dealt with, but never bothered to look into.

shopt -s histappend works perfectly and now I don't have to wonder which session I'm getting my history from :)

Thanks!

, Mike

Kalmi
Quoth Kalmi on June 29, 2008 @ 3:55 PM PDT

I find it a lot less confusing this way: alias u='history -n' export PROMPT_COMMAND="history -a;"

nickwe
Quoth nickwe on February 26, 2009 @ 11:15 PM PST

Great, it bother me for so long, never took the time to search for a work around and I just get to this page looking for something else and now I've resolve an old problem!

Many Thanks!

Hand Histories
Quoth Hand Histories on March 26, 2009 @ 9:55 AM PDT

I am still new to Linux and I had the same problem as I use many terminals simultaneously. The combination of shopt -s histappend and export PROMPT_COMMAND="history -n; history -a" is very handy. Thank you, I had been looking for something like that for a while. I mean in linux shell, you always have to look at your history, it saves a lot of time.

cde
Quoth cde on April 09, 2009 @ 2:21 PM PDT

I had the same problem. Well, half the problem. I just added history -a instead of both -n and -a. With both, you have the problem Brian explained, and after years of being accustomed to one history file per active terminal, it would be too annoying to change that around. By just adding the -a to the bashrc, you can do -n only when you need to.

Balan K
Quoth Balan K on June 06, 2009 @ 10:54 AM PDT

Thanks for the information. I am just looking for a way to manage my multiterminal environment. shopt -s histappend should be handy tool.

ontario motels
Quoth ontario motels on June 07, 2009 @ 3:49 AM PDT

Great, it bother me for so long, never took the time to search for a work around and I just get to this page looking for something else and now I've resolve an old problem! always keep th good work going on your side.. congrats

Jacob Godserv
Quoth Jacob Godserv on October 22, 2009 @ 1:21 AM PDT

Oh my gosh! This is brilliant! I wish I knew about this years ago!

erick
Quoth erick on October 22, 2009 @ 3:27 AM PDT

Thanks!

Doyle Brunson
Quoth Doyle Brunson on November 08, 2009 @ 12:14 PM PST

Excellent, it bothered me for a long time, but I never took the time to search for a work around and I just get to this page looking for something else and now I have solved my old problem.

Thanks s ton

æon
Quoth æon on January 12, 2010 @ 7:52 PM PST

Thanks!!!

Amos
Quoth Amos on June 11, 2010 @ 9:04 AM PDT

Thanks for the tip.

A couple of small corrections

  1. It's not necessary to "export" PROMPT_COMMAND (i.e. it doesn't need to be set as an environment variable). It's enough to do "PROMPT_COMMAND='history -n'".

  2. If you want to keep running both then you can just set PROMPT_COMMAND to "history -an". Although after reading the comments here I agree that the -n is going to be annoying.

Jeremydei
Quoth Jeremydei on July 26, 2010 @ 8:15 AM PDT

Nice! Thanks I needed the reload part. This is a priceless setup. I've also written an article about this on my blog ^^

PaulC
Quoth PaulC on January 10, 2012 @ 1:39 AM PST

Awesome. Thanks for this. Our sysadmins "auto logout" users idle for more than 15 minutes, they do so by killing your shell! So this is essential or you can literally lose days of histroy (if you've been running top to stop the auto-logout and then forget it once!).

PROMPT_COMMAND="history -a"

And I can do history -n if I wish manually.

I think it's a little bizarre that "history -n" does not appear in history.

PhilC
Quoth PhilC on March 25, 2012 @ 5:09 PM PDT

Zsh has this built in

Gonzih
Quoth Gonzih on March 25, 2012 @ 5:16 PM PDT

Try zsh, also you can try oh-my-zsh for quick start.

Alex
Quoth Alex on March 25, 2012 @ 5:54 PM PDT

I don't read the history into the current shell once it's started, to prevent crossing over commands between concurrent terminals (I have about 50 across my three virtual desktops just this second...), but I do append it out to a (huge) global history file.

However, if you do like have a shared global command memory - including cross-pollinating all your terminals continuously - you might consider adding use of HISTTIMEFMT, embedding the current tty in it along with the timestamp, and then having a user-defined history function (say, "hme") that can run history (with the time and your tty coming free with "history"), filter for the current tty, then strip out the timestamps if you don't need them.

Only problem is I don't quite see a way to bind M-p or something to a bash function, to allow the lines considered as candidates to be drawn from a user-defined functions output. You could have a function to toggle which view of history is loaded however, while still saving each new command to the global one. Which could be fairly awesome.

Anonymous
Quoth Anonymous on March 25, 2012 @ 7:00 PM PDT

Shouldn't PROMPT_COMMAND be "history -n; history -a" ?

Jack
Quoth Jack on March 25, 2012 @ 8:35 PM PDT

Thanks for this. Ubuntu had already added the shopt -s histappend for me, but setting the PROMPT_COMMAND has been really useful! I agree that I only want to set it to history -a (and do history -n manually) because I find it weird if I can see all the previous commands I've executed in a particular terminal, but pressing up doesn't give them back to me...

Ste Richards
Quoth Ste Richards on March 25, 2012 @ 8:37 PM PDT

This causes unexpected behaviour on FreeBSD 7.2 :(

Luca
Quoth Luca on March 26, 2012 @ 1:39 AM PDT

This is a good solution. I think you should also take a look at bash_eternal_history.

http://www.debian-administration.org/articles/543

Alexander
Quoth Alexander on April 04, 2012 @ 8:30 AM PDT

Surely it must be "history -a; history -n" (instead of -n, then -a), otherwise the current shell's session's pending history will be erased before it gets the chance to be appended. Seems to be confirmed with some casual testing.

Brian
Quoth Brian on April 04, 2012 @ 8:51 AM PDT

You're right Alexandar, should be -a, then -n. My .bashrc had it the correct way.

I'm somewhat surprised this blog post is still getting traffic after 5 years. Other things in this post may be out of date/inaccurate. I've actually been using ZSH for the past couple years.

too
Quoth too on April 04, 2012 @ 4:16 PM PDT

Yes, it's helpful. And set PROMPT_COMMAND="history -a;" which mention in comments is great too!

Alexander
Quoth Alexander on April 05, 2012 @ 9:52 AM PDT

In this case the reason for the sudden traffic is probably that your post was linked to from Server Density's blog.

Speak your Mind

You can use Markdown in your comment.
Email/URL are optional. Email is only used for Gravatar.

Preview