[olug] sort -u vs uniq

[Noel Leistad]

Thanks, always knew I needed easy help using keys...

cat file| sort -n | sort -u|wc -l
cat file|sort -n | uniq | wc -l
cat file|sort -t. -k1,1n -k2,2n -k3,3n -k4,4n -u |wc -l

ALL give the same result, something just a little different about sort 
and combined flags, -n -u, -nu -un all give the same, but diffrent 
result than above.

Mystery not solved, but quickly becoming un-interesting... (gonna call 
that signs of a large list rather than an impairment of advancing years).

Noel

On 03/13/2017 11:08 AM, Herb Wolfe wrote:
> I have a e-book on shell scripting that covers almost this exact situation.
> It is sorting the hosts file, so doesn't use unique, but the rest is the
> same.
>
> sort -t. -k1,1n -k2,2n -k3,3n -k4,4n -u
>
> Where -t. Is used to specify the . as the delimiter, while -k1,1n indicates
> that the first key is field 1 to 1, inclusive, and is numeric, rather than
> alphabetical.
>
> On Mar 13, 2017 10:23 AM, "Noel Leistad" <noel at metc.net> wrote:
>
> I'm trying to get a list of uniq IP addresses from a log file. I have a
> list of ALL IP addresses. Using sort -nu and sort -n | uniq give me 2
> different lists.
>
> A stare and compare make me think that sort -nu  only considers the first 2
> octets as significant. RTFM of the sort man page indicates sort honors
> LC_COLLATE.
>
> <appear uninformed>
> LC_COLLATE isn't in env, so I'm assuming it's set at build/compile time
> when building sort or in the c libraries someplace?
> </appear uninformed -- hardly, stupid probably better tag... and not
> closed.>
>
> Could this be why the sort -u and uniq return differing output? I don't see
> anyplace to specify "how much" to consider significant when running sort.
> Anyone care to offer thoughts?
>
> Thanks.
>
>
> Noel
>
> _______________________________________________
> OLUG mailing list
> OLUG at olug.org
> https://lists.olug.org/mailman/listinfo/olug
> _______________________________________________
> OLUG mailing list
> OLUG at olug.org
> https://lists.olug.org/mailman/listinfo/olug

-- 
Noel Leistad
Network Administrator
Marne & Elk Horn Telephone Company
4242 Main Street
Elk Horn, IA 51531

http://www.metc.net