It won’t come as a surprise by now that we love shell progress bars.
When we are copying big files it would be nice to have an ETA, progress bar and some feedback, but the cp command doesn’t provide us with this.
Of course, we can use rsync to get some feedback of the process, but if we want a progress bar probably the most simple way is using pv.
pv, or pipe viewer is a very neat shell tool that allows you to monitor a pipe. It is invoked in a way that is similar to the cat command. You can pipe to it and monitor the throughput of the data that goes through the pipe
cat srcfile | pv > dstfile
You can also open a file directly with it, just like cat. This is in fact how you would “copy” a file using pv
pv srcfile > dstfile
But that syntax is not very nice to use, plus it is limited to a single file at a time. I wanted to use it with whole directories, recursively so I came up with a little wrapper, with the astoundingly original name cpv.
Usage
Invoke just like cp. No options. Obviously requires that you have pv installed.
cpv file_or_dir_src file_or_dir_dst
That’s it. It is really a very simple wrapper but it is handy.
It is also useful when copying from or to network mounts to measure the transfer speed.
Installation
Add the code to your .zshrc or .bashrc. You can do this in one line, but inspect it first so you don’t have to trust me blindly.
wget https://raw.githubusercontent.com/nachoparker/cpv/master/cpv.sh -O - >> ~/.zshrc # ~/.bashrc
More cool stuff with pv
You can do much more that copying. You can get data about just about anything that uses files or pipes.
You can for instance monitor that netcat transfer
pv srcfile | nc -w 1 somewhere.com 3000
Or the creation of that tarball
tar -cf - dir | pv > tarball.tar
Or a docker export
docker export minidebian_container | pv > minidebian_export.tar
Or that ssh pipe
tar -cfz - dir | pv | ssh pi@raspi "cat > ~/dir.tar.gz"
Or that database dump
mysqldump -u root --single-transaction nextcloud | pv > db-backup.sql
You can flash a SD card with progress (even though we finally have dd status=progress )
pv NextCloudPi.img > /dev/sdc; sync
Zero a drive
pv < /dev/zero > /dev/sda
It can even track open file descriptors, so you can monitor the operation afterwards the process started
mysqldump -u root --single-transaction nextcloud > db-backup.sql pv -d $(pidof mysqldump)
Code
#!/bin/bash # pv wrapper that imitates cp usage # # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com> # GPL licensed (see end of file) * Use at your own risk! # # Usage: # cpv file_or_dir_src file_or_dir_dst # # More details at ownyourbits.com function cpv() { local DST=${@: -1} # last element local SRC=( ${@: 1 : $# - 1} ) # array with rest of elements # checks type pv &>/dev/null || { echo "install pv first"; return 1; } [ $# -lt 2 ] && { echo "too few args" ; return 1; } # special invocation function cpv_rename() { local SRC="$1" local DST="$2" local DSTDIR="$( dirname "$DST" )" # checks if [ $# -ne 2 ]; then echo "too few args" ; return 1; fi if ! [ -e "$SRC" ]; then echo "$SRC doesn't exist" ; return 1; fi if [ -d "$SRC" ]; then echo "$SRC is a dir" ; return 1; fi if ! [ -d "$DSTDIR" ]; then echo "$DSTDIR does not exist"; return 1; fi # actual copy echo -e "\n$SRC ? $DST" pv "$SRC" >"$DST" } # special case for cpv_rename() if ! [ -d "$DST" ]; then cpv_rename "$@"; return $?; fi; # more checks for src in "${SRC[@]}"; do local dst="$DST/$( basename "$src" )" if ! [ -e "$src" ]; then echo "$src doesn't exist" ; return 1; elif [ -e "$dst" ]; then echo "$dst already exists"; return 1; fi done # actual copy for src in "${SRC[@]}"; do if ! [ -d "$src" ]; then local dst="$DST/$( basename "$src" )" echo -e "\n$src ? $dst" pv "$src" > "$dst" else local dir="$DST/$( basename "$src" )" mkdir "$dir" || continue local srcs=( $src/* ) cpv "${srcs[@]}" "$dir"; fi done unset cpv_rename } # License # # This script is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This script is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this script; if not, write to the # Free Software Foundation, Inc., 59 Temple Place, Suite 330, # Boston, MA 02111-1307 USA
Check out other shell articles
Or just use rsync –info=progress2
does that provide a progress bar? not last time I checked
Hello nachoparker,
Which distro os linux you are using?
I like GUI desktop , may you help me sent the name ?
Thanks
Hi, I use Arch Linux with plain openbox… quite minimal stuff really
Hello,
how can I copy the whole directory recursively?
it copies recursively by default
Nice code, thanks for sharing. I use pv for copying (backing up) VM’s in KVM. I like having the progress bar, because some of the bigger ones, I question if it is hung up or still working away.