Re: Syntax Question...

[Dennis Williamson]

On Mon, Aug 15, 2011 at 2:45 AM, Linda Walsh <bash@tlinx.org> wrote:
>
>
>
> ` Linda Walsh wrote:
>>
>> The latest error I got is a a simple type -- most of them probably are,
>> with that many
>> lines of code in ~3-4 weeks, there's bound to be -- trouble is I'm
>> stubborn sometimes
>> about 'punishing myself'' when I mess up...which isn't always productive!
>>  ;-)
>
> Sometimes typo's used to work -- i.e. sometimes characters get
> randomly deleted out of my source files --
>
> Win7 drops keys randomly, and I don't always notice the effects....
>
> anyway, you wanted....
>
> I wouldn't regard them as a finished product...
>
>
>
> #!/bin/bash -exu
>
> # copyright(c) Linda Walsh 2011 ;
> # create_snap_rdiff Version: 0.1; initial writing
> # indended for eventual gnuish-open source license
>
>
> #set -xuhmeB
>
> export GLOBAL_SET_OPS="${-:-}"
> shopt -s expand_aliases extglob shift_verbose xpg_echo
> export
> PS4='>>${BASH_SOURCE:+${BASH_SOURCE[0]}}#${LINENO}${FUNCNAME:+(${FUNCNAME[0]})}>
> '
> this="create_snap_rdiff"
>
> progpath="$0"
> prog="${progpath##*/}"
> progdir="${progpath%/*}"
> export progpid=$$
>
> #base_mp="/home"
> shopt -s expand_aliases extglob
>
> alias sub=function
> alias unless='if !'
> typeset -xr sub unless
>
> declare -a SAVE_ARGS=( "$@" )
> typeset -xr snapdir='snapdir'
>
> # in shell: white is black and black is white:
> export b_false=1
> export b_true=0
> export Debug=$b_false  Verbose=$b_true  Silent=$b_false
> export Quiet=$b_false  Force=""  loglevel=7
> export base_mp=""
>
> . "$progdir/snaplib.sh"
> read progdir prog path export_blob <<<$(env_init "$progdir" "$prog" "$this")
> export PATH="$path"
> eval "$export_blob"
>
> #echo "progdir=$progdir, prog=$prog, path=$path"
> #echo "exportblob"
>
> export PATH="$path"
> eval "$export_blob"
>
>
> if [[ $#<1 ]]; then
>        errx 50 "Need mount_path of fs to create snap_rdiff from"
> fi
>
> set "${SAVE_ARGS[@]}"
>
> declare -a args
> declare -a blob
>
> function process_args { # {{{
>        DebugPush ProcArgs
>        args=( "${@}" )
>        local -i argi=0
>        # Arg Processing...
>        while (($#>0)); do
>                a="$1";shift;
>                if [[ $a == "-v" || $a == "--verbose" ]]; then
>                        ((Verbose=b_true,loglevel<6?loglevel=6:0,0))
>                elif [[ $a == "-d" || $a == "--debug" ]]; then
>                        echo "Debug Set"
>                        ((Debug=b_true,loglevel<7?loglevel=7:0,0))
>                elif [[ $a == "-f" ]] ; then
>                        Force="$a";shift;continue;
>                elif [[ $a =~ ^--force=.* ]]; then
>                        Force="${a#*=}"
>                elif [[ $a == "--quiet" ]]; then
>                        ((Quiet=b_true,loglevel>2?loglevel=2:0,0))
>                elif [[ $a == "--silent" ]]; then
>                        ((Silent=b_true,loglevel=0))
>                elif [[ $a =~ --?.* ]]; then
>                        { echo "Unknown option \"$a\", use --help for help";
> } >&1
>                        exit -1
>                else
>                        args[$argi]="$a"
>                        ((++argi))
>                fi
>        done
>        if [[ ${#args[*]} -ge 1 ]]; then
>                echo -e "${args[@]}"
>        else
>                echo
>        fi
>        if [[ -z "$base_mp" && ${#args[*]}<1 ]]; then
>                errx 51 "Need mount_path of fs to create snap_rdiff from"
>        fi
>        DebugPop
> }
>
> args=( $(process_args "$@" ) )
> set ${args[@]}
>
> ### allowed 'features' (to be set/queried, must be defined 1st)
> # (to prevent feature-misnaming/mispelling)
> #
>
> def_Provide base_mp base_vg base_lv base_mp_diff
> def_Provide clean_diff_dir
> def_Provide daily_snap
> def_Provide empty_snapvol
> def_Provide filled_snapvol
> def_Provide formatted_snapvol full_diff_dir
> def_Provide labeled_diff_dir
> def_Provide new_snapshot_wanted
> def_Provide osnap_mp osnap_vg osnap_lv osnap_devpath
> #def_Provide oldest_active_snapshot osnap_mp osnap_vg osnap_lv osnap_devpath
> def_Provide oldest_active_snapshot
> def_Provide root_dev
> def_Provide snap_vol snap_mp snap_mp_dir snap_copy_complete
>
>
>
> ## provides base_mp
> base_mp="${1:?}"
> add_provides "base_mp"
>
> #}}}
>
> #devel options
> export i_skip_mnt_clean_diff_dir=0
> export i_skip_rsync=0
> export i_skip_today_check=0
> export i_skip_vol_create=0
> export i_skip_new_snap=0
>
>
>
> #needed init
> ##
> ## provides base_vg base_lv root_dev
> ##
> read base_vg base_lv <<<$(get_vg_lv_from_mp "$base_mp")
> root_dev="$(get_devnum / )"
> verb "root_dev=$root_dev, base_vg=$base_vg, base_lv=$base_lv"
> add_provides "base_vg" "base_lv" "root_dev"
>
>
>
>
> ##
> ## provides: new_snapshot_wanted
> ## requires: base_mp
> ## May-use:  Force
> ##
> # won't return unless override is set or no backup today
> ((i_skip_today_check)) || {
>        if have_dep "base_mp"; then
>                chk_today_snap "$base_mp" "$Force"
>                add_provides "new_snapshot_wanted"
>        fi
> }
>
>
>
> base_mp_diff="$base_mp.diff"
> add_provides base_mp_diff
>
>
>
>
>
>
> # Attempt to read snap-label info set after a successful rsync
> read osnap_mp osnap_vg osnap_devpath <<<$(find_unsaved_snap_data
> "$base_mp_diff")
> if [[ ${osnap_mp:-} && ${osnap_vg:-} && ${osnap_devpath:-} ]]; then
>        osnap_vg="$(getVG_by_MntDir "$osnap_devpath")" && {
>                add_provides osnap_vg
>        }
>        add_provides labeled_diff_dir
>        add_provides full_diff_dir
>        add_provides osnap_mp osnap_lv osnap_devpath
>        ## now check to see if this diff has been copied to corresponding
> snapdir!
>        check_snap_copy_complete "$osnap_mp"
> fi
>
>
>
>
> if ! have_dep labeled_diff_dir || have_dep snap_copy_complete   ; then
>        # if doesn't exist we clean the 'diff' vol, and generate a new one
>        # using 'rsync'
>
>        # clean diff dir
>        ##
>        ## provides: clean_diff_dir
>        ## depends: !unsaved_snap_data_check
>        ## removes: full_diff_dir
>        ##
>        ((!i_skip_mnt_clean_diff_dir)) && {
>        if ( ! have_dep "labeled_diff_dir" || have_dep snap_copy_complete ) ;
> then
>                        mnt_n_clean_diff_dir "$base_mp_diff" && {
>                                add_provides "clean_diff_dir"
>                                rm_provides "full_diff_dir"
>                        }
>                fi
>        }
>
>        # find oldest active snapshot
>        ##
>        ## provides: oldest_active_snapshot
>        ## provides: osnap_mp osnap_vg osnap_lv osnap_devpath
>        ##
>        unset osnap_mp osnap_vg osnap_lv
>        read osnap_mp osnap_vg osnap_lv osnap_devpath \
>                                <<<$( find_oactv_snapshot "$base_lv" )
>        if [[ ${osnap_mp:-} && ${osnap_vg:-} && ${osnap_lv:-} ]]; then
>                add_provides "oldest_active_snapshot"
>                add_provides "osnap_mp" "osnap_vg" "osnap_lv" "osnap_devpath"
>        fi
>        check_snap_copy_complete "${osnap_mp:-}"
>
>        msg_level $_Info_ && {
>                if have_dep snap_copy_complete; then
>                        info "Have snap_copy_complete"
>                else
>                        info "snap_copy_complete is not set"
>                fi
>        }
>
>
>        ##
>        ## provides: daily_snap
>        ## depends: base_mp
>        ##
>        if ((!i_skip_new_snap)); then
>                if have_dep "base_mp" "new_snapshot_wanted"; then
>                        create_basemp_snap "$base_mp" && {
>                                add_provides "daily_snap"
>                                rm_provides "new_snapshot_wanted"
>                        }
>                fi
>        fi
>
>
>
>        ##
>        ## create diff vol
>        ##
>        ## provides: full_diff_dir
>        ## provides: labeled_diff_dir
>        ## Depends: osnap_mp osnap_vg osnap_vg
>        ##
>        if ((!i_skip_rsync)) ; then
>                if have_dep -v osnap_mp base_mp base_mp_diff ; then
>
>                        # note, '**' must be quoted so bash's globstar won't
> expand it
>                        declare -a exclude_pats=(
>                                "--exclude='.recycle/**'"
>           "--exclude='BLISS/law/**'"
>                                "--exclude='BLISS/athenae/**'"
>  "--exclude='BLISS/law.V2/**'"
>                                "--exclude='BLISS/martin/**'"
>                                "--exclude='CPAN-ishtar-build-cache/**'" )
>
>                        rsync_snapdiffs_to_base_mp_diff \
>                         '\rsync_snapdiffs_to_base_callback' \
>                         "$osnap_mp/." "$base_mp_diff"                  \
>                         "$base_mp"  "${exclude_pats[@]}" && {
>                                add_provides "full_diff_dir"
>                        }
>                fi
>        fi
>
>
>
>
>        ## write label on diff-vol if rdiff finished ok
>        ##
>        ## write_osnap_identifier_file
>        ## Provides: labeled diff dir"
>        ## Depsends: base_mp_diff full_diff_dir osnap_mp osnap_lv
> osnap_devpath
>        ##
>        if have_dep -v base_mp_diff full_diff_dir osnap_mp osnap_lv
> osnap_devpath;
>                then
>                write_osnap_identifier_file \
>                        "$base_mp_diff" "$osnap_mp" "$osnap_lv"
> "$osnap_devpath" && {
>                        add_provides "labeled_diff_dir"
>                }
>        fi
>
>
>
>
>        ## if have labeled_diffdir, then delete snapshot it came from
>        ##
>        ## destroy_old_snapshot_by_mp_n_devpath
>        ## Provides:
>        ## Depends: labeled_diff_dir oldest_active_snapshot
>        ## Deletes:
>        if -v have_dep labeled_diff_dir osnap_mp osnap_devpath; then
>                destroy_old_snapshot_by_mp_n_devpath "$osnap_mp"
> "$osnap_devpath" && {
>                        rm_provides "oldest_active_snapshot"
>                }
>        fi
>
>
> ### Getting here means we have a 'diff' vol that we need to move
> ### to a downwardly sized static vol that will contain the diff
> ### (a static snap)
>
>
> ##
> ## provides snap_vol
> ## provides osnap_devpath
> ## requires base_mp_diff base_vg osnap_lv osnap_vg
> ##
> if have_dep -v base_mp_diff base_vg osnap_lv osnap_vg ;then
>        nextents="$(get_nextents_for_space_on_vg_w_minfree  \
>                                                        "$base_mp_diff"
> "$base_vg" 10 )"
>        $_sudo  $_lvcreate -l $nextents -r 256 -n "$osnap_lv"  "$osnap_vg" &&
> {
>                add_provides snap_vol
>                osnap_devpath="/dev/$osnap_vg/$osnap_lv"
>                add_provides osnap_devpath
>        }
> fi
>
>
>
>
> ## provides xfs_fs_for_oldest_snap_dir
> ## depends-on: oldest_snap_dir_lv
> if have_dep osnap_lv base_lv osnap_devpath; then
>        osnap_xfs_label="${osnap_lv#"$base_lv-20"}"
>        $_sudo make_xfs_snap_archive_vol \
>                "${osnap_xfs_label:0:11}" "$osnap_devpath" && {
>                add_provides empty_snapvol
>                add_provides formatted_snapvol
>        }
> fi
>
>
>
>
> ## requires: base_mp base_lv osnap_lv
> ## provides snap_mp
> if have_dep base_mp base_lv osnap_lv; then
>        obase_mp="/o${base_mp#/}"
>        datedir="${osnap_lv#$base_lv-}"
>        snap_mp="$base_mp/$snapdir/@GMT-$datedir"
>        add_provides snap_mp
>        if [[ -e $snap_mp ]]; then
>                if [[ -d $snap_mp ]]; then
>                        if mp_dev_is_not_same_as_2nd_fs $snap_mp
> $base_mp/$snapdir; then
>                                $_umount $snap_mp ||
>                                        errx 54 "something mounted at
> $snap_mp and won't unmount"
>                        fi
>                        $_sudo $_rmdir $snap_mp || {
>                                errx 55 "won't mount on non-empty dir at
> $snap_mp"
>                        }
>                else
>                        if [[ $Force =~ rm_osnapdir_contents ]]; then
>                                $rm -fr $snap_mp
>                        fi
>                        if [[ -e $snap_mp ]]; then
>                                errx 52 "$snap_mp is either not a dir, or
> can't be removed"
>                        fi
>                fi
>        fi
> fi
>
>
>
> ## provides: snap_mp_dir
> ## requires: snap_mp
> have_dep snap_mp && $_sudo mkdir $snap_mp && {
>        add_provides snap_mp_dir
> } || errx 56 "Can't create mount point $snap_mp"
>
>
>
>
> #provides mounted_new_snap
> #depends: snap_mp_dir formatted_snapvol
> if have_dep snap_mp_dir formatted_snapvol osnap_devpath snap_mp; then
>        $_sudo $_mount $osnap_devpath $snap_mp && {
>                add_provides mounted_snapvol
>        }
> fi
>
> if have_dep snap_mp; then
>        check_snap_copy_complete "$snap_mp"
> fi
>
> ## provides: snap_copy_complete
> ## requires: mounted_new_snap
>
> if have_dep base_mp_diff snap_mp; then
>        if ! have_dep snap_copy_complete; then
>                $_sudo $_cp -a $base_mp_diff/. $snap_mp/. && {
>                        $_sudo $_touch "$snap_mp/snap_copy_complete"
>                        add_provides snap_copy_complete
>                }
>        fi
> fi
>
>
> function check_clean_diff_dir {
>        local base_mp_diff="${1:?}"
>        [[ -e $base_mp_diff/clean_diff_dir ]] &&
>                add_provides clean_diff_dir
> }
>
> if have_dep base_mp_diff ; then
>        check_clean_diff_dir "$base_mp_diff"
> fi
>
>
> ## provides: clean_diff_dir
> ## requires: snap_copy_complete base_mp_diff
>
> if have_dep snap_copy_complete base_mp_diff;then
>        mnt_n_clean_diff_dir "$base_mp_diff" && {
>                $_sudo $_touch "$base_mp_diff/clean_diff_dir"
>                add_provides clean_diff_dir
>        }
> fi
>
> fi
>
> # vim: ts=2 sw=2 fdm=marker fen number
>
> #!/bin/bash
> #
> # tracing active for now...
>
> # today_has_snap: 0.1 initial writing,
> # support routine for 'create_snap_rdiff'
> # copyright (c) 2011 Linda A. Walsh <la_walsh <at> tlinx <dot> org)
> # indended for eventual gnuish-open source license
>
>
> # return true if today has snap,
> # return non-zero in any failure (including existence of today-snap)
>
> this="today_has_snap"
> progpath="$0"
> prog="${progpath##*/}"
> progdir="${progpath%/*}"
> if [[ ${progdir:0:1} != '/' ]]; then
>        preprog="$PWD"
>        progdir="$preprog/$progdir"
> fi
>
> [[ $prog =~ bash ]] && prog="$this"
>
> . "$progdir/snaplib.sh"
> read progdir prog path export_blob <<<$(env_init "$progdir" "$prog" "$this")
> export PATH="$path"
> eval "$export_blob"
>
>
>
> if [[ $# < 1 ]]; then
>                                echo "$prog: need mount point"
>                                exit -2;
> fi
> Vol="$1"
> VG="$(getVG_by_MntDir $Vol)"
> LV="$(getLV_by_MntDir $Vol)"
>
> if [[ -z $VG || -z $LV ]]; then
>        echo "$Vol not found (VG=$VG), (LV=$LV)"
>        exit -3
> fi
>
> SnapDir=snapdir
> Dev=/dev/$VG/$LV
> Prefix="snapdir/@GMT-"
> snap_today="$(date +%Y.%m.%d-)"
> snap_prefix="$Vol/$Prefix$snap_today*"
> today_snaps="$('ls' -1 ${snap_prefix} 2>/dev/null |tr "\n" " " )"
> if [[ -n $today_snaps && $today_snaps != $snap_prefix ]]; then
>        echo "yes"
>        exit 0
> fi
> exit 1
> # vim: ts=2 sw=2
>
>
>

Greg already covered some important points and I'll add a few more.

As has been said before, don't use "bash -e".

How do I determine the location of my script?
http://mywiki.wooledge.org/BashFAQ/028

To process your arguments, use "case" instead of a string of if/else.
You could instead use "getopts" if you only needed to support short
options (or if you're brave you could use the external getopt(1), but
there are versions that work and those that don't). See
http://mywiki.wooledge.org/BashFAQ/035.

In your arg processing, you copy $@ into $arg, but then you assign
each argument to elements in $arg individually as you iterate over $@.
Why? Also, it's usually not necessary to maintain an index variable
and use shift in a while loop. Use "for arg; do" which uses an implied
$@.

I have no idea what "if [[ ${var:-} ..." is intended to do. That
parameter expansion says "if $var is null or unset, substitute a null
string" which has no effect. Just use "if [[ $var ..." which means
exactly the same thing.

"if -v have_dep" in one place (which probably results in a "command
not found error" and "if have_dep -v" in another. I think something
got transposed.

"mp_dev_is_not_same_as_2nd_fs" - Ack!

I could go on (and on), but this is making me tired. I didn't intend
for you to post an entire script (and its subscripts). Rather, I
intended that you post a problematic section of code and specific
goals for that portion. Then we could help you make that bit of code
accomplish its intended result (while simultaneously avoiding an XY
problem).

> today_snaps="$('ls' -1 ${snap_prefix} 2>/dev/null |tr "\n" " " )"

This one is so bad, I saved it for last. Ack! Pfffft! Wait, what? Why?
What the? Huh?

-- 
Visit serverfault.com to get your system administration questions answered.