Bash methods

Bash Script Methods - Quick View  updates

I'm going to compile a few ideas here that I find useful for developing quick script solutions, particularly for my newer CI approaches.

This page may be a bit messy and you may need to search for various terms to find the code you are after. This is just a notepad for my use.

Get value between two strings

master.html snippet:

    <?xml version="1.0" encoding="UTF-8" standalone="yes"?><searchResults start-index="0" max-result="10" size="210"><searchResults id="1.0.314" type="deploymentVersion"><searchEntity xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="deploymentVersionSearchResult" id="1.0.314"><name>1.0.314</name></searchEntity></searchResults><searchResults id="1.0.313" type="deploymentVersion"><searchEntity xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="deploymentVersionSearchResult" id="1.0.313"><name>1.0.313</name></searchEntity></searchResults><searchResults id="1.0.311" type="deploymentVersion"><searchEntity xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="deploymentVersionSearchResult" id="1.0.311"><name>1.0.311</name></searchEntity></searchResults><searchResults id="1.0.309" type="deploymentVersion"><searchEntity xmlns:xsi="http://www.w3.org/2001/XMLSchema-

R_num=$(grep '<name>' master_.html | awk 'BEGIN {FS="<name>"} {print $2}' | awk 'BEGIN {FS="</name>"} {print $1}')
echo $R_num

return value:

    1.0.314

With wget, quote any URLs with dodgy caharacters (which may not break a browser) - like & which is used as a bash character for background:

    wget --header="Authorization: Basic bmdvZG............WhtZWUwQWk=" --header "Cookie: atl.xsrf.token=${atl_token}; JSESSIONID=61716178C187AE5B6DB63EEAFFBAB173" 'https://bamboo.adm.xxxxcloud.net/rest/api/latest/search/versions?searchTerm=&start-index=0&chronologicalOrder=true&max-results=10&deploymentProjectId=65142785&branchKey=CRBS-CRBSB&_=1472221371698' -O master.html

NOTE: the � �s around the URL because of the &�s in there. Other characters can also create such problems (without the quotes I get 400 bad request).

     

Multiple conditions with if

Got this of the internet but I do find it useful, the () for grouping &&�s:

   my_error_flag=1
   my_error_flag_o=1
   if [ $my_error_flag -eq 1 ] || [ $my_error_flag_o -eq 2 ] || ([ $my_error_flag -eq 1 ] && [ $my_error_flag_o -eq 2 ]); then
         echo "$my_error_flag"
   else
       echo "no flag"
   fi

  
While Loop example

   #check jtl files for pass mark and return value:

   ls -all *.jtl | awk '{print $9}' > jtl_files
   echo get number of jtl entries
   jtl_nf=$(wc -l jtl_files | awk '{print $1}')
   echo number = ${jtl_nf}

   passmark=0
   START=1
   END=${jtl_nf}
   echo start = $START
   echo end = $END

   ## save $START, just in case if we need it later ##
   i=$START
   while [[ $i -le $END ]]
   do
         echo "$i"
        
        
         tail -100 jtl_files | head -$i | tail -1
         jtl_var=$(tail -100 jtl_files | head -$i | tail -1)

         if grep -q 's="false"' ${jtl_var}
         then
               echo ---------------------------------------
               echo FAIL - last jtl file sets value to fail
               echo ---------------------------------------
               passmark=1
         fi
                    
         ((i = i + 1))
        
        
   done
              
   exit ${passmark}
 

Count lines in log files

tail access.log | awk '{print $5}' | awk 'BEGIN {FS="/"} {print $2}' | wc -l

wget with cookies to see secure pages:

Example from the net:

    wget --no-cookies --header "Cookie: <name>=<value>" --post-data 'tab=search&name=+&personsubmit=Rechercher&keywords=&keywordsoption=allmine' https://moodle.unistra.fr/message/index.php

 

Example from my secure jenkins server (tested but edited for security):

    wget --no-cookies --header "Cookie: screenResolution=1920x1080; JSESSIONID.3359c4fb=907de103b5713ef1d6355e68850bc; JSESSIONID.9c30b717=80ad4fa2c0869bfa93a74b6aa8be; screenResolution=1920x1080; ACEGI_SECURITY_HASHED_REMEMBER_ME_COOKIE=bmdvZGZyZXTQ3NDY0MjQ1NjEwMjo4ZDA5OTUxYTUyOWI4MWE0Zj hkOTI5MTRlMTMwZGRiNQ==" http://work-deploy01/jenkins/view/All/ -O all.html

(I used Fiddler to grab the cookie value from a browser trace)

Get current status from a jenkins job:

    R_num=$(grep all4_run_test_5tps all.html | awk 'BEGIN {FS="tooltip=\""} {print $2}' | head -1 | awk 'BEGIN {FS="\""} {print $1}')

    echo $R_num

Variables inside quotes

    echo "$VARIABLE"
    echo "${VARIABLE}"
    Use of single quotes is the main issue. According to the
    Bash Reference Manual

 

Print lines from a file from the point of finding a phrase

    start=$(grep -n personalisation.user_datalist-perf info.html | head -1 | awk 'BEGIN {FS=":"} {print $1}')
    echo $start
    ((end = start + 20))
    sed -n "65,${end}p" info.html > check
    cat check

Trim characters from the start and end of a line

remove character from the end of a line:

echo "hello world" | rev | cut -c5- | rev
# hello w
But I used it to trim lines in a file so that's why it looks awkward. The real use was:

cat somefile | rev | cut -c5- | rev
cut only gets you as far as trimming from some starting position, which is bad if you need variable length rows. So this solution reverses (rev) the string and now we relate to its ending position, then uses cut as mentioned, and reverses (again, rev) it back to it's original order.

hence - to take off the last character off each line:
$ tail VTS\ data\ 2016-09-13\(08_42_54\)_test_13th_sept_2016_09_43.csv | awk -F ';' 'BEGIN {FS=","} {print $2,$3}'
to:
$ tail VTS\ data\ 2016-09-13\(08_42_54\)_test_13th_sept_2016_09_43.csv | awk -F ';' 'BEGIN {FS=","} {print $2,$3}' | rev | cut -c2- | rev

cat VTS\ data\ 2016-09-13\(08_42_54\)_test_13th_sept_2016_09_43.csv | awk -F ';' 'BEGIN {FS=","} {print $2,$3}' | rev | cut -c2- | rev > one_2016_09_43.csv

cat VTS\ data\ 2016-09-13\(13_47_18\)_test_15tps_13th_sept_2016_14_47.csv | awk -F ';' 'BEGIN {FS=","} {print $2,$3}' | rev | cut -c2- | rev > two_2016_14_47.csv

 

Grep for an ip number (in an LR scenario file)

grep -o '{[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' ucs_one_hour.lrs | cut -c 2-

(note: the first �{� is part of the pattern we are looking for - first character on the line. NOT part of the grepping technique)

Get the first ip:

grep -o '{[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' ucs_one_hour.lrs | cut -c 2- | tail -10 | head -1

 

Get the second ip:

grep -o '{[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' ucs_one_hour.lrs | cut -c 2- | tail -10 | head -2 | tail -1

 

Check URL return values

$ wget --spider http://dt06.channel.local/

    Spider mode enabled. Check if remote file exists.

    --2016-09-26 15:11:54--  http://dt06.channel.local/

    Resolving dt06.channel4.local (dt06.channel.local)... ***.25.12.***

    Connecting to dt06.channel.local (dt06.channel.local)|***.25.12.***|:80... connected.

    HTTP request sent, awaiting response... 200 OK

    Length: unspecified [text/html]

    Remote file exists and could contain further links,

    but recursion is disabled -- not retrieving.

 

$ curl --head http://dt06.channel.local/

    HTTP/1.1 200 OK

    Date: Mon, 26 Sep 2016 14:11:59 GMT

    Server: Apache/2.4.10 (Win32) OpenSSL/1.0.1i PHP/5.6.3

    X-Powered-By: PHP/5.6.3

    Access-Control-Allow-Origin: *

    Set-Cookie: o=c0afcdcc652a1c1b963a036e37790b649b1feb03; expires=Tue, 26-Sep-2017 14:11:59 GMT; Max-Age=31536000; path=/

    Content-Type: text/html; charset=UTF-8

 

Replace a string in a file throughout, using a variable

Note: you have to use double quotes to keep the variable working, see above:

    base=$(grep BaseResultsName ucs_one_hour.lrs | awk 'BEGIN {FS="="} {print $2}')
    echo $base

    sed -i -- "s/${base}/res001/g" ucs_one_hour.lrs

    #and need to change this as well, if it�s set to zero:

    sed -i -- "s/AutoSetResults=0/AutoSetResults=1/g" ucs_one_hour.lrs

 

 

 

 

[Home] [About (CV)] [Contact Us] [JMeter Cloud] [webPageTest] [_64 images] [asset moniitor] [Linux Monitor] [Splunk ETL] [Splunk API] [AWS bash] [LR Rules OK] [LR Slave] [LR CI Graphs] [LoadRunner CI] [LR CI Variables] [LR Bamboo] [LR Methods] [LR CI BASH] [Bash methods] [Jenkins V2] [Streaming vid] [How fast] [Finding Issues] [Reporting] [Hand over] [VB Scripts] [JMeter tips] [JMeter RAW] [Dynatrace] [Documents] [FAQ] [Legal]

In the Cartesian Elements Ltd group of companies