The accepted answer is really useful, but I found that it was slow with a large-ish database. I believe it also limits your options when joining the data.
My method now is to pull everything into SQLite (using a combination of csvkit and ogr2ogr):
csvsql --db sqlite:///myjoindb.db --insert myjoincsv.csvogr2ogr -append -f "SQLite" myjoindb.db myjoinshp.shp
Then join everything and create a shapefile out of it:
ogr2ogr -f "ESRI Shapefile" -sql "SELECT csv.*, shp.* FROM myjoinshp shp INNER JOIN myjoincsv csv ON csv.joinfield = shp.joinfield" joined_output.shp myjoindb.db