17Nov/20

Create Github Repo On The Go From The Shell – Github API

2020-11-17Programmingkossboss

Sometimes you start coding something and you don’t yet realize if it will be a project worth sharing on github. We don’t always think about this as we begin coding. This isRead More…

29Oct/20

Python + Guitar + All Notes

2020-10-29Music, Programmingkossboss

I came across this medium article about python and guitar strings and plotting scales. It has an interesting Jupyter notebook to work with, allowing to plot scales for all of the chords.Read More…

23Oct/20

Python remove duplicates similar to bash uniq + sort

2020-10-23Linux, Programmingkossboss

# citing: https://www.reddit.com/r/Python/comments/jfx7wy/how_to_quickly_remove_duplicates_from_a_list/

# Let's make some duplicates (one million numbers between 1 and 100)

from random import randrange
DUPLICATES = [ randrange(100) for _ in range(1_000_000) ]

### METHOD 1 ### 

# Not very efficient - Similar to "sort | uniq" but the original order is kept

unique = []
for element in DUPLICATES:
    if element not in unique:
        unique.append(element)
print(unique)

# Output:
[95, 62, 73, 27, 89, 88, 31, 50, 26, 97, 13, 18, 54, 64, 17, 94, 72, 65, 83, 14, 40, 91, 1, 21, 22, 52, 61, 15, 7, 80, 24, 42, 37, 96, 74, 9, 59, 44, 66, 20, 6, 84, 39, 81, 2, 67, 32, 77, 36, 5, 8, 47, 55, 82, 60, 35, 33, 75, 57, 68, 29, 11, 23, 86, 69, 99, 25, 78, 76, 19, 12, 92, 10, 90, 16, 3, 51, 4, 28, 0, 71, 46, 79, 85, 58, 48, 93, 56, 53, 98, 87, 30, 45, 63, 70, 38, 41, 43, 34, 49]

### METHOD 2 ###

# Very efficient - Similar to "sort | uniq"

list(set(DUPLICATES))

# This works because sets contain unique items by definition

# Output:
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]

### METHOD 3 ###

# Very efficient but order preserved
# Not sure which bash function this is similar to
# Resembles "sort | uniq" but with the original order preserved
# But sets are unordered! What if we need to preserve the order?
# Use this dict.fromkeys() trick!
list(dict.fromkeys(DUPLICATES))

# Output:
[95, 62, 73, 27, 89, 88, 31, 50, 26, 97, 13, 18, 54, 64, 17, 94, 72, 65, 83, 14, 40, 91, 1, 21, 22, 52, 61, 15, 7, 80, 24, 42, 37, 96, 74, 9, 59, 44, 66, 20, 6, 84, 39, 81, 2, 67, 32, 77, 36, 5, 8, 47, 55, 82, 60, 35, 33, 75, 57, 68, 29, 11, 23, 86, 69, 99, 25, 78, 76, 19, 12, 92, 10, 90, 16, 3, 51, 4, 28, 0, 71, 46, 79, 85, 58, 48, 93, 56, 53, 98, 87, 30, 45, 63, 70, 38, 41, 43, 34, 49]

### METHOD 4 ###

# Older Python - Similar to "uniq | sort"
# But it only works for Python 3.6 and above
# For Python 2.7 and 3.0-3.5, use OrderedDict:from collections import OrderedDict
list(OrderedDict.fromkeys(DUPLICATES))

#### METHOD 5 ###

# Crunch duplicates as they appear and show count
# Similar to bashes "uniq -c"
# I will show you how to do "uniq -c" on a string
# Just change the input to a list and it will work on a list

# METHOD 5.1 #

from itertools import groupby
input = "Raaanndommmm Leetters"
for i,v in groupby(input):
   print(len(list(v)),i)

# Output:
1 R
3 a
2 n
1 d
1 o
4 m
1
1 L
2 e
2 t
1 e
1 r
1 s

# METHOD 5.2 #

# Or you can return a list for an output:

l=[]
for i,v in groupby(input):
   l.append(str(len(list(v)))+" "+str(i))
print(l)

# Output:
['1 R', '3 a', '2 n', '1 d', '1 o', '4 m', '1  ', '1 L', '2 e', '2 t', '1 e', '1 r', '1 s']

#  METHOD 5.3 #

# Same solution as a one liner:

print([ str(len(list(v)))+" "+str(i) for i,v in groupby(input)])

# Output:
['1 R', '3 a', '2 n', '1 d', '1 o', '4 m', '1  ', '1 L', '2 e', '2 t', '1 e', '1 r', '1 s']

### METHOD 6 ###

# Crunch out duplicates but dont show count

# METHOD 6.1 #

from itertools import groupby
input = "Raaanndommmm Leetters"
for i,v in groupby(input):
   print(str(i))

# Output:
R
a
n
d
o
m

L
e
t
e
r
s

# METHOD 6.2 #

l=[]
for i,v in groupby(input):
   l.append(str(i))
print(l)

# Output:
['R', 'a', 'n', 'd', 'o', 'm', ' ', 'L', 'e', 't', 'e', 'r', 's']

# METHOD 6.3 #

print([ str(i) for i,v in groupby(input)])
# Output:

['R', 'a', 'n', 'd', 'o', 'm', ' ', 'L', 'e', 't', 'e', 'r', 's']

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

# citing: https://www.reddit.com/r/Python/comments/jfx7wy/how_to_quickly_remove_duplicates_from_a_list/

# Let's make some duplicates (one million numbers between 1 and 100)

from random import randrange

DUPLICATES = [ randrange(100) for _ in range(1_000_000) ]

### METHOD 1 ###

# Not very efficient - Similar to "sort | uniq" but the original order is kept

unique = []

for element in DUPLICATES:

if element not in unique:

unique.append(element)

print(unique)

# Output:

[95, 62, 73, 27, 89, 88, 31, 50, 26, 97, 13, 18, 54, 64, 17, 94, 72, 65, 83, 14, 40, 91, 1, 21, 22, 52, 61, 15, 7, 80, 24, 42, 37, 96, 74, 9, 59, 44, 66, 20, 6, 84, 39, 81, 2, 67, 32, 77, 36, 5, 8, 47, 55, 82, 60, 35, 33, 75, 57, 68, 29, 11, 23, 86, 69, 99, 25, 78, 76, 19, 12, 92, 10, 90, 16, 3, 51, 4, 28, 0, 71, 46, 79, 85, 58, 48, 93, 56, 53, 98, 87, 30, 45, 63, 70, 38, 41, 43, 34, 49]

### METHOD 2 ###

# Very efficient - Similar to "sort | uniq"

list(set(DUPLICATES))

# This works because sets contain unique items by definition

# Output:

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]

### METHOD 3 ###

# Very efficient but order preserved

# Not sure which bash function this is similar to

# Resembles "sort | uniq" but with the original order preserved

# But sets are unordered! What if we need to preserve the order?

# Use this dict.fromkeys() trick!

list(dict.fromkeys(DUPLICATES))

# Output:

### METHOD 4 ###

# Older Python - Similar to "uniq | sort"

# But it only works for Python 3.6 and above

# For Python 2.7 and 3.0-3.5, use OrderedDict:from collections import OrderedDict

list(OrderedDict.fromkeys(DUPLICATES))

#### METHOD 5 ###

# Crunch duplicates as they appear and show count

# Similar to bashes "uniq -c"

# I will show you how to do "uniq -c" on a string

# Just change the input to a list and it will work on a list

# METHOD 5.1 #

from itertools import groupby

input = "Raaanndommmm Leetters"

for i,v in groupby(input):

print(len(list(v)),i)

# Output:

1 R

3 a

2 n

1 d

1 o

4 m

1 L

2 e

2 t

1 e

1 r

1 s

# METHOD 5.2 #

# Or you can return a list for an output:

l=[]

for i,v in groupby(input):

l.append(str(len(list(v)))+" "+str(i))

print(l)

# Output:

['1 R', '3 a', '2 n', '1 d', '1 o', '4 m', '1 ', '1 L', '2 e', '2 t', '1 e', '1 r', '1 s']

# METHOD 5.3 #

# Same solution as a one liner:

print([ str(len(list(v)))+" "+str(i) for i,v in groupby(input)])

# Output:

['1 R', '3 a', '2 n', '1 d', '1 o', '4 m', '1 ', '1 L', '2 e', '2 t', '1 e', '1 r', '1 s']

### METHOD 6 ###

# Crunch out duplicates but dont show count

# METHOD 6.1 #

from itertools import groupby

input = "Raaanndommmm Leetters"

for i,v in groupby(input):

print(str(i))

# Output:

# METHOD 6.2 #

l=[]

for i,v in groupby(input):

l.append(str(i))

print(l)

# Output:

['R', 'a', 'n', 'd', 'o', 'm', ' ', 'L', 'e', 't', 'e', 'r', 's']

# METHOD 6.3 #

print([ str(i) for i,v in groupby(input)])

# Output:

['R', 'a', 'n', 'd', 'o', 'm', ' ', 'L', 'e', 't', 'e', 'r', 's']

16Mar/20

Coronavirus Dashboard – covid19.py

2020-03-16Programming, Web Developingkossboss

Here is my take on a coronavirus dashboard that uses daily updated json data from countries. To view the covid19.py dashboard click here: covid19-normal.html To view the log scale covid19.py dashboard clickRead More…

14Feb/16

Multiple ways to pass arguments to Python

2016-02-14Programmingkossboss

Below is examples of passing arguments to python via regular argument assignment, and tuples and *tuples, and dictionaries and **dictionaries. Here is my gist link for it: https://gist.github.com/anonymous/271d6160cdd90a482b40 (older) Not covering all methods (forRead More…

11Feb/16

Computer Networking – End to End Argument

2016-02-11Networking, Programmingkossboss

Rule of thumb: when implementing a function or program, you should implement it at the end points. Quote: End-to-end connectivity is a property of the Internet that allows all nodes of the network toRead More…

29Jan/16

GIT pulling 1 file – reverting file to last commit – file undo

2016-01-29Programmingkossboss

So you changed a file and now you want the original back (the original from the last commit). You want to undo the changes. I do it like this: git checkout HEADRead More…

06Oct/15

Searching Source Code with GREP or ACK

2015-10-06Linux, Programmingkossboss

Articles to check out: * https://www.digitalocean.com/community/tutorials/how-to-install-and-use-ack-a-grep-replacement-for-developers-on-ubuntu-14-04 * http://beyondgrep.com/why-ack/ * http://readwrite.com/2010/11/10/how-to-search-your-source-with GREP (rinIE) or EGREP (rinI) the all round search utility

grep -rinIE "match word" /folder
grep -rinIE "match word" /folder
egrep -rinI "match word" /folder

grep -rinIE "match word" /folder

egrep -rinI "match word" /folder

NOTE: the grep article recommend -inIEr, but thats hard to remember,Read More…

15Apr/15

How to plot/graph data in a file with Python using Linux – Bonus: Dates on X axis & Trend Line

2015-04-15Linux, Other, Programmingkossboss

This article is probably worth reading twice due to my style of writing (sorry). Read the comments as they have alot of informational text. I tried to make them understandable to aRead More…

14Dec/14

tohex & tobin – python program – convert files to hexadecimal/hex or binary output

2014-12-14Linux, Programmingkossboss

If you want to convert a program to hexdecimal or binary out, here is two programs to do so. First get your pre-reqs:

apt-get update
apt-get install python
apt-get install pip
pip install bitstring

apt-get update

apt-get install python

apt-get install pip

pip install bitstring

Note: these programs would work outside of linux,Read More…

infotinks

My Notes, Articles & Guides for Linux, Windows and Networking.

Category Archives: Programming

Create Github Repo On The Go From The Shell – Github API

Python + Guitar + All Notes

Python remove duplicates similar to bash uniq + sort

Coronavirus Dashboard – covid19.py

Multiple ways to pass arguments to Python

Computer Networking – End to End Argument

GIT pulling 1 file – reverting file to last commit – file undo

Searching Source Code with GREP or ACK

How to plot/graph data in a file with Python using Linux – Bonus: Dates on X axis & Trend Line

tohex & tobin – python program – convert files to hexadecimal/hex or binary output