WordPress/Linux Bash CLI Notes Engine Facility

I created this process to improve my record keeping while working on a Linux bash shell. It allows for one to take notes while on the command line as well as publish to WordPress for tidy and organized record keeping.

Features:

  • Posts logs to WordPress from any Linux hosts.
  • Log all CLI commands to syslog globally.
  • Record literal notes to syslog.
  • Privately post logs to WordPress via CLI.
  • Posting of Subject from CLI to WP posts.
  • Auto insert [hostname] into WordPress post subject.
  • Auto remove undesired data from WP posts.
  • Auto insert [shortcode] into WordPress posts.

Roadmap:

  • Auto redact sensitive data on CLI WP post actions.
  • Record file changes to syslog (fork tripwire).
  • Post Categories to WordPress posts from CLI.
  • Post Tags to WordPress posts from CLI.

Requirements:

  • A self-hosted WordPress Installation.
  • The free Twenty Fifteen Theme.
  • FOSS Postie Plugin by Wayne Allen.
  • FOSS SyntaxHilighter Evolved by Alex Mills.
  • A pop3 mailbox (Postie to pull & DELETE emails).
  • Logger via the util-linux package.
  • Mailx via the mail-utils package.
  • Rsyslog via rsyslog package.
  • A bit of troysio microcode.

Steps:

  • Install WordPress.
  • Activate Twenty Fifteen Theme.
  • Download/Activate SyntaxHilighter Evolved in plugins.
  • Download/Activate Postie within plugins area.
  • Configure Postie to fetch and posts PRIVATELY text only.
  • Install mail-utils and util-linux packages on all Linux hosts.
  • Create your CLI logging syslog service.
  • Drop the following troysio microcode into e.g. /usr/local/bin/yourio and make it executable.
  • #!/bin/sh
    # Troy Perkins
    # https://troys.io
    # Note: Make sure to remove all "\"es from Shortcode lines before executing. I had to place them in order for this code to be published here due to surrounding bash shortcode in this post.
    
    #Host
    host=`hostname`;
    
    #Pop3 mailbox;
    mailb="your@email.com";
    
    #Content
    file='/var/log/commands.log';
    #file='/var/log/test.log';
    
    #Shortcode
    shrtstart='\[\bash]';
    shrtend='\[\/\bash]';
    
    { echo $shrtstart; cat $file | cut -d':' --complement -s -f1,2,3,4 | sed 's/\ [^\ ]*$//'; echo $shrtend; } | uniq | mailx -s "[$host] $1" $mailb
    

Usage:
To take manual notes type a # sign then your note then hit enter:

root@crm:# # Here is an example of a manual note on the command line.

To send notes to WordPress:

root@crm:# troysio "Here is my subject"

Example Output:

vi /usr/bin/troysio
vi /usr/bin/troysio-redact
troysio "testing new cut"
cat /var/log/commands.log | cut -d':' --complement -s -f1,2,3,4
vi /usr/bin/troysio
vi /usr/bin/troysio-redact
troysio "testing new cut"
troysio uniq
vi /usr/bin/troysio
troysio uniq
troysio | uniq
vi /usr/bin/troysio
vi /usr/bin/troysio-redact
troysio "testing uniq"
vi /usr/bin/troysio
troysio "testing uniq"
cat -A /var/log/commands.log
cp /usr/bin/troysio /usr/bin/troysio-test
vi /usr/bin/troysio-test
troysio-test | cat -A
troysio-test | sub(/\r$/,"") | cat -A
troysio-test | cat -A
# Here is an example of a manual note on the command line.
# Hope you enjoy your note taking helper.
# Cheers.
troysio "here is my subject"
# WordPress post made with above command with title [crm] < host and "here is my subject" on the title of post.
# Joy

To view the actual post, which I’ve changed from Private to Public just for you go here: https://troys.io/crm-here-is-my-subject/

Purge Windows 10 of its useless bloatware

Uninstall individual and most crap apps in Windows 10 using PowerShell.

To start, search for PowerShell in your Start menu, right-click on it and select the option (Run as Administrator).” Or, open a Command Prompt as user administrator, type start Powershell and press the Enter button.

List pre-installed Microshit apps

Get-AppxPackage | ft Name, PackageFullName -AutoSize

List all systemwide apps

Get-AppxPackage -AllUsers | ft Name, PackageFullName -AutoSize

Remove all pre-installed apps by diarrhea Microcrap

Get-AppxPackage -AllUsers | Remove-AppxPackage

Stop pre-provisioning aka on-boarding from happening again by sneaky Gate’s Microdung when new account is added

Get-AppXProvisionedPackage -online | Remove-AppxProvisionedPackage –online

Note: Let the last command run for a while, it seems like it’s stuck in a loop but it is not. Let it complete. Rinse and reaping a few times doesn’t hurt either.

Find following Registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\

Create new key inside the Windows key folder. Right-click the Windows key, choose New > Hit Key. Name the new key CloudContent.
Create a new value inside that new key. Right-click the CloudContent key and select New > Select DWORD (32-bit) Value. Name the new value DisableWindowsConsumerFeatures.

Double-click the new DisableWindowsConsumerFeatures value to open its properties window. Change the value from 0 to 1 in the Value data box and then click the ok button, then reboot.

The above actions will remove even those sticky apps like Xbox, Skype, Candy Crush and more. Final registry edition is suppose to block them from being reconstructed and leaking back in. Thanks for wasting everyone’s precious time Bill. Quit vomiting on your latests releases asshole.

rsync volume to volume with speed in consideration

rsync -axHAWXS --numeric-ids --info=progress2 /source/ /destination/

What the options do:
-a : all files, with permissions, etc..
-v : verbose, mention files
-x : stay on one file system
-H : preserve hard links (not included with -a)
-A : preserve ACLs/permissions (not included with -a)
-X : preserve extended attributes (not included with -a)
-W : avoid calculating deltas/diffs of the files (for speed)
-S : to consider sparse/tiny files

–numerical-ids : avoids the overhead of mapping UID/GID values (more speed)
–info=progress2 : avoid even more overhead by showing overall progress

Block countries, networks and VoIP blacklists combining Netfilter’s iptables, ipset and voipbl.org

Make ipset a service,keeping block lists alive and surviving power cycles:

vi /lib/systemd/system/ipset.service

[Unit]
Description=IPSET VoIP Blacklist
Before=netfilter-persistent.service
ConditionFileNotEmpty=/etc/ipset/voipbl

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/sbin/ipset -exist -file /etc/ipset/voipbl restore
ExecStop=/sbin/ipset -file /etc/ipset/voipbl save

[Install]
WantedBy=multi-user.target
Alias=ipset.service

systemctl daemon-reload
systemctl enable ipset

vi /usr/local/bin/voipbl.sh

#!/bin/bash

exec 5> >(logger -t $0)
BASH_XTRACEFD="5"
PS4='$LINENO: '
set -x

SHELL=/bin/bash
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin
MAILTO=alerts@email.com

get_voiprbl='http://voipbl.org/update/'
get_arinonly='http://voipbl.org/update/?wn[]=arin'

if [ -e '/etc/ipset/voipbl.txt' ]
then
  echo "/etc/ipset/voipbl.txt file exists."
  echo "Checking timestamp and size..."
  olddt=`stat -c '%y' /etc/ipset/voipbl.txt`
  oldsize=`ls -lh /etc/ipset/voipbl.txt | cut -d" " -f5`
else
  echo "/etc/ipset/voipbl.txt not found."
  echo "Touching file for first run..."
  touch /etc/ipset/voipbl.txt
  olddt=`stat -c '%y' /etc/ipset/voipbl.txt`
  oldsize=`ls -lh /etc/ipset/voipbl.txt | cut -d" " -f5`
fi

if [ -e '/etc/ipset/arinonly.txt' ]
then
  echo "/etc/ipset/arinonly.txt file exists."
  echo "Checking timestamp and size..."
  olddtarin=`stat -c '%y' /etc/ipset/arinonly.txt`
  oldsizearin=`ls -lh /etc/ipset/arinonly.txt | cut -d" " -f5`
else
  echo "/etc/ipset/arinonly.txt not found."
  echo "Touching file for first run..."
  touch '/etc/ipset/arinonly.txt'
  olddtarin=`stat -c '%y' /etc/ipset/arinonly.txt`
  oldsizearin=`ls -lh /etc/ipset/arinonly.txt | cut -d" " -f5`
fi

echo "Downloading VoIPBL GLOBAL IP network shuns."
wget -qO - $get_voiprbl -O /etc/ipset/voipbl.txt
echo "Downloading US/CA ARIN networks only lists."
wget -qO - $get_arinonly -O /etc/ipset/arinonly.txt
echo ""
newdt=`stat -c '%y' /etc/ipset/voipbl.txt`
newdtarin=`stat -c '%y' /etc/ipset/arinonly.txt`
newsize=`ls -lh /etc/ipset/voipbl.txt | cut -d" " -f5`
newsizearin=`ls -lh /etc/ipset/arinonly.txt | cut -d" " -f5`
echo "voipbl.txt file differentials:"
echo "old: $olddt SIZE: $oldsize"
echo "new: $newdt SIZE: $newsize"
echo ""
echo "arinonly.txt file differentials:"
echo "old: $olddtarin SIZE: $oldsizearin"
echo "new: $newdtarin SIZE: $newsizearin"
echo ""
echo "Creating hash lists in memory..."
ipset create -exist tmp_voipbl hash:net
ipset create -exist tmp_arin hash:net
ipset create -exist voipbl hash:net
ipset create -exist arinonly hash:net
echo ""

if [ -e '/etc/ipset/tmp_voipbl' ]; then
  echo "/etc/ipset/tmp_voipbl file exists."
  echo "Preparing hash lists for swaping..."
else
  echo "/etc/ipset/tmp_voipbl not found."
  echo "Touching file for first run..."
  touch '/etc/ipset/tmp_voipbl'
fi

cp "/dev/null" "/etc/ipset/tmp_voipbl"
cp "/dev/null" "/etc/ipset/tmp_arin"
echo ""
echo "Parsing new downloads..."

for voipblist in `tail -n +2 /etc/ipset/voipbl.txt`; do
  echo add tmp_voipbl $voipblist >> /etc/ipset/tmp_voipbl
done

for arin in `tail -n +2 /etc/ipset/arinonly.txt`; do
  echo add tmp_arin $arin >> /etc/ipset/tmp_arin
done

# swap the temp ipsets for the live ones
ipset flush tmp_voipbl
ipset flush tmp_arin
ipset restore < /etc/ipset/tmp_voipbl
ipset restore < /etc/ipset/tmp_arin
ipset flush voipbl
ipset flush arinonly
ipset swap tmp_voipbl voipbl
ipset swap tmp_arin arinonly
echo ""
ipset save tmp_voipbl -f /etc/ipset/tmp_voipbl
ipset save tmp_arin -f /etc/ipset/tmp_arin
ipset destroy tmp_voipbl
ipset destroy tmp_arin
ipset save voipbl -f /etc/ipset/voipbl
ipset save arinonly -f /etc/ipset/arinonly
echo "List inventory in RAM and in use by Netfilter:"
echo ""
echo "`ipset list -t`"

# log the file modification time for use in minimizing lag in cron schedule
moredt=`date`;
logger -p cron.notice "IPSet: voipbl updated as of: $moredt" ;

#Check if rules in iptables
if ! $(/sbin/iptables -w --check INPUT -m set --match-set voipbl src -j LOGNDROP > /dev/null 2>&1); then
  /sbin/iptables -I INPUT 1 -m set --match-set voipbl src -j LOGNDROP
fi

if ! $(/sbin/iptables -w --check INPUT -m set --match-set arinonly src -j DROP > /dev/null 2>&1); then
  /sbin/iptables -I INPUT 1 -m set --match-set arinonly src -j DROP
fi

echo "";
echo "Netfilter IPSet rules updated and reloaded into RAM";
echo "Successful completion...";
echo "Bye Bye...";

crontab -e

0 1 * * * /usr/local/bin/voipbl.sh

Via email:

/etc/ipset/voipbl.txt file exists.
Checking timestamp and size...
/etc/ipset/arinonly.txt file exists.
Checking timestamp and size...
Downloading VoIPBL GLOBAL IP network shuns.
Downloading US/CA ARIN networks only lists.

voipbl.txt file differentials:
old: 2018-11-25 11:00:21.502886736 -0600 SIZE: 868K
new: 2018-11-26 01:00:19.904743514 -0600 SIZE: 879K

arinonly.txt file differentials:
old: 2018-11-25 11:00:30.438886415 -0600 SIZE: 631K
new: 2018-11-26 01:00:21.100743556 -0600 SIZE: 651K

Creating hash lists in memory...

/etc/ipset/tmp_voipbl file exists.
Preparing hash lists for swaping...

Parsing new downloads...

List inventory in RAM and in use by Netfilter:

Name: voipbl
Type: hash:net
Revision: 5
Header: family inet hashsize 32768 maxelem 65536
Size in memory: 1394616
References: 1

Name: arinonly
Type: hash:net
Revision: 5
Header: family inet hashsize 16384 maxelem 65536
Size in memory: 792280
References: 1

Netfilter IPSet rules updated and reloaded into RAM
Successful completion...

OpenWRT R11E-LTE-US

AT E0 V1 

AT+CFUN=1,1 

AT E0 V1 

AT+GMI 

AT+GMM 

AT+GMR 

AT+CGSN 

AT+CFUN=4 

AT+CMEE=1 

AT+CREG=2 

AT+CGREG=2 

AT+CEREG=2 

AT+CGEREP=2,0 

AT+CMGF=0 

AT+CNMI=1,1,0,1,0 

AT+CFUN=1 

AT+CPIN? 

AT+CPMS="SM","SM","SM" 

AT+CFUN? 

AT+CNUM 

AT+CIMI 

AT+CPIN? 

AT+COPS=0 

AT+CGDCONT=1,"IP","vzwinternet" 

AT$QCPDPP=1,0 

AT+CEREG=2 

AT+COPS? 

AT+CFUN? 

AT+CSQ 

AT+COPS? 

AT+CFUN? 

QAT$QCRSRP? 

QAT$QCRSRQ? 

at$ecmcall=1 

at$ecmcall? 

AT+CGCONTRDP=1

[crm] Here is my subject

The following is example output from my WordPress/Linux CLI Notes Engine Project.

# This is an example of my WordPress/Linux CLI Notes Engine output.
# Its reference can be found https://troys.io/wordpress-linux-cli-notes-engine
# Initial title was "Private: [crm] Here is my subject" because its default posts are private.  I've changed it to public within WordPress so you can see this example.
# Enjoy
exit
sudo postconf -e "home_mailbox = Maildir/"
sudo postconf -e "mailbox_command = "
sudo systemctl restart postfix
env
mail
exit
ls -al
cd /home/troy
ls -al
ls -al Maildir
mail troy
ls -al Maildir
tail -f /var/log/mail.log
dpkg-reconfigure postfix
systemctl restart postfix
tail -f /var/log/mail.log
cd /etc/postfix
ls -al
vi main.cf
vi /etc/aliases
systemctl restart postfix
exit
cat /var/log/commands.log | sed 's/ .$//'
cat /var/log/commands.log | sed 's/ .*$//'
cat /var/log/commands.log | sed 's/\.[^\.]*$//'
cat /var/log/commands.log
cat /var/log/commands.log | sed 's/\ [^\ ]*$//'
# Test this line ########
# Test this line ##### ##
# Test this line $%(^)%1!
cat /var/log/commands.log | sed 's/\ [^\ ]*$//'
cat /var/log/commands.log
vi /usr/bin/troysio
cp /usr/bin/troysio /usr/bin/troysio-redact
vi /usr/bin/troysio
troysio "testing last word chop"
vi /usr/bin/troysio
cat /var/log/commands.log | cut -d':' --complement -s -f1,2,3,4
vi /usr/bin/troysio
vi /usr/bin/troysio-redact
troysio "testing new cut"
cat /var/log/commands.log | cut -d':' --complement -s -f1,2,3,4
vi /usr/bin/troysio
vi /usr/bin/troysio-redact
troysio "testing new cut"
troysio uniq
vi /usr/bin/troysio
troysio uniq
troysio | uniq
vi /usr/bin/troysio
vi /usr/bin/troysio-redact
troysio "testing uniq"
vi /usr/bin/troysio
troysio "testing uniq"
cat -A /var/log/commands.log
cp /usr/bin/troysio /usr/bin/troysio-test
vi /usr/bin/troysio-test
troysio-test | cat -A
troysio-test | sub(/\r$/,"") | cat -A
troysio-test | cat -A
# Here is an example of a manual note on the command line.
# Hope you enjoy your note taking helper.
# Cheers.
troysio "here is my subject"
# WordPress post made with above command with title [crm] < host and "here is my subject" on the title of post.
# Joy