Bypassing ssh Permissions Too Open Warning
This trick works for Linux servers and also if you are using WSL2 on Windows to run Ubuntu (or whatever *nix OS) and you are trying to utilize ssh keys with yourRead More…
This trick works for Linux servers and also if you are using WSL2 on Windows to run Ubuntu (or whatever *nix OS) and you are trying to utilize ssh keys with yourRead More…
I can never remember the order of arguments for the ln command (link command). The easiest way to think about it is to rethink of the “cp” (copy command) instead of “cpRead More…
This github page, https://github.com/andreafabrizi/Dropbox-Uploader, has the dropbox-uploader tool which I use to backup content from servers (Linux, Mac, etc) without having to sync my Dropbox content to the local disk on yourRead More…
Xargs is useful to run in parallel. Its parallel processing is very efficient. Read this post about its efficiency and this one about basic commands. Below, is my favorite way to runRead More…
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 |
# citing: https://www.reddit.com/r/Python/comments/jfx7wy/how_to_quickly_remove_duplicates_from_a_list/ # Let's make some duplicates (one million numbers between 1 and 100) from random import randrange DUPLICATES = [ randrange(100) for _ in range(1_000_000) ] ### METHOD 1 ### # Not very efficient - Similar to "sort | uniq" but the original order is kept unique = [] for element in DUPLICATES: if element not in unique: unique.append(element) print(unique) # Output: [95, 62, 73, 27, 89, 88, 31, 50, 26, 97, 13, 18, 54, 64, 17, 94, 72, 65, 83, 14, 40, 91, 1, 21, 22, 52, 61, 15, 7, 80, 24, 42, 37, 96, 74, 9, 59, 44, 66, 20, 6, 84, 39, 81, 2, 67, 32, 77, 36, 5, 8, 47, 55, 82, 60, 35, 33, 75, 57, 68, 29, 11, 23, 86, 69, 99, 25, 78, 76, 19, 12, 92, 10, 90, 16, 3, 51, 4, 28, 0, 71, 46, 79, 85, 58, 48, 93, 56, 53, 98, 87, 30, 45, 63, 70, 38, 41, 43, 34, 49] ### METHOD 2 ### # Very efficient - Similar to "sort | uniq" list(set(DUPLICATES)) # This works because sets contain unique items by definition # Output: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99] ### METHOD 3 ### # Very efficient but order preserved # Not sure which bash function this is similar to # Resembles "sort | uniq" but with the original order preserved # But sets are unordered! What if we need to preserve the order? # Use this dict.fromkeys() trick! list(dict.fromkeys(DUPLICATES)) # Output: [95, 62, 73, 27, 89, 88, 31, 50, 26, 97, 13, 18, 54, 64, 17, 94, 72, 65, 83, 14, 40, 91, 1, 21, 22, 52, 61, 15, 7, 80, 24, 42, 37, 96, 74, 9, 59, 44, 66, 20, 6, 84, 39, 81, 2, 67, 32, 77, 36, 5, 8, 47, 55, 82, 60, 35, 33, 75, 57, 68, 29, 11, 23, 86, 69, 99, 25, 78, 76, 19, 12, 92, 10, 90, 16, 3, 51, 4, 28, 0, 71, 46, 79, 85, 58, 48, 93, 56, 53, 98, 87, 30, 45, 63, 70, 38, 41, 43, 34, 49] ### METHOD 4 ### # Older Python - Similar to "uniq | sort" # But it only works for Python 3.6 and above # For Python 2.7 and 3.0-3.5, use OrderedDict:from collections import OrderedDict list(OrderedDict.fromkeys(DUPLICATES)) #### METHOD 5 ### # Crunch duplicates as they appear and show count # Similar to bashes "uniq -c" # I will show you how to do "uniq -c" on a string # Just change the input to a list and it will work on a list # METHOD 5.1 # from itertools import groupby input = "Raaanndommmm Leetters" for i,v in groupby(input): print(len(list(v)),i) # Output: 1 R 3 a 2 n 1 d 1 o 4 m 1 1 L 2 e 2 t 1 e 1 r 1 s # METHOD 5.2 # # Or you can return a list for an output: l=[] for i,v in groupby(input): l.append(str(len(list(v)))+" "+str(i)) print(l) # Output: ['1 R', '3 a', '2 n', '1 d', '1 o', '4 m', '1 ', '1 L', '2 e', '2 t', '1 e', '1 r', '1 s'] # METHOD 5.3 # # Same solution as a one liner: print([ str(len(list(v)))+" "+str(i) for i,v in groupby(input)]) # Output: ['1 R', '3 a', '2 n', '1 d', '1 o', '4 m', '1 ', '1 L', '2 e', '2 t', '1 e', '1 r', '1 s'] ### METHOD 6 ### # Crunch out duplicates but dont show count # METHOD 6.1 # from itertools import groupby input = "Raaanndommmm Leetters" for i,v in groupby(input): print(str(i)) # Output: R a n d o m L e t e r s # METHOD 6.2 # l=[] for i,v in groupby(input): l.append(str(i)) print(l) # Output: ['R', 'a', 'n', 'd', 'o', 'm', ' ', 'L', 'e', 't', 'e', 'r', 's'] # METHOD 6.3 # print([ str(i) for i,v in groupby(input)]) # Output: ['R', 'a', 'n', 'd', 'o', 'm', ' ', 'L', 'e', 't', 'e', 'r', 's'] |
iostat service time is a very useful metric when analyzing disk performance and finding bottlenecks service time is essentially the inverse of IOPs so if an operation takes 1ms to service, thenRead More…
function displaytime { local T=$1 local D=$((T/60/60/24)) local H=$((T/60/60%24)) local M=$((T/60%60)) local S=$((T%60)) (( $D > 0 )) && printf ‘%d days ‘ $D (( $H > 0 )) && printf ‘%dRead More…
We need to run ifconfig and look for the status line. Remember this about ifconfig output: * if “status: active“, the port is linked up and sending traffic (now you just needRead More…
If your system gzips and rotates your logs. You can use this trick to search thru all of them. # zgrep -i — “SEARCHTERM” $(ls -1Str /var/log/messages*) Another way # zgrep -iRead More…
Got info from here: https://stackoverflow.com/questions/18363833/how-can-i-list-upnp-server-renderer-in-command-line-console-mode-on-linux Get the Upnp test tools
1 |
# apt-get install gupnp-tools |
Example 1: Find all Upnp devices (note their target names start with urn:…. It will hint at what service it is, ex: Layer3Forwarder,Read More…