Discussion:
what's the current volume of USENET?
(too old to reply)
Salvador Mirzo
2024-12-18 20:40:36 UTC
Permalink
I'm looking for any notion of how busy USENET is right now, so the
number of servers and the number of articles posted by month is enough
for me. I found

https://top1000.anthologeek.net/2024/11/participants.txt

which is for November, 2024. At the end, we find

--8<-------------------------------------------------------->8---
Total number of articles seen: 238521959
Total number of participants: 75
Total number of articles seen, after equalizing: 28443388.
--8<-------------------------------------------------------->8---

What is the meaning of ``after equalizing''?
--
Thanks!
noel
2024-12-19 13:42:37 UTC
Permalink
Post by Salvador Mirzo
I'm looking for any notion of how busy USENET is right now, so the
number of servers and the number of articles posted by month is enough
for me. I found
Number of servers you'll never know, that top1000 depends on admins
bothering to configure and send stats, for every server that does there's
probably 20 that don't - I know we don't.

As for articles, text groups, in past 22.5 hours (since last daily
maintenance run) we've received just over 21K articles, at about 51mb.
So grows about 1.5gb a month, roughly. far cry from years gone by where
we'd see that and more in just one day, but, some people like the eye
candy bloatware of html on forums now for some reason i'll never
understand :)
D
2024-12-19 15:15:49 UTC
Permalink
Post by noel
Post by Salvador Mirzo
I'm looking for any notion of how busy USENET is right now, so the
number of servers and the number of articles posted by month is enough
for me. I found
Number of servers you'll never know, that top1000 depends on admins
bothering to configure and send stats, for every server that does there's
probably 20 that don't - I know we don't.
As for articles, text groups, in past 22.5 hours (since last daily
maintenance run) we've received just over 21K articles, at about 51mb.
So grows about 1.5gb a month, roughly. far cry from years gone by where
we'd see that and more in just one day, but, some people like the eye
candy bloatware of html on forums now for some reason i'll never
understand :)
it's only a theory, but if the infinite universe is perfect, and whole
galaxies are in fact self-balancing systems, then by keeping imperfect
humans out of the loop (i.e., never understanding) that way everything
remains perfectly in balance . . . so for every action there can be an
equal and opposite action, forever in keeping with the right-hand rule

as for why some might prefer bells, buzzers and whistles to plain-text
content . . . could be distraction, deflection, a means of escape from
the ineffable realm of imagination, in favor of the material existence
as prisoners of gravity in time, the divine right of kings to wage war
in perpetuity . . . they love to live by the sword, it's their destiny

not exactly on-topic, but usenet seems more tolerant than social media
Salvador Mirzo
2024-12-21 12:28:20 UTC
Permalink
Post by noel
Post by Salvador Mirzo
I'm looking for any notion of how busy USENET is right now, so the
number of servers and the number of articles posted by month is enough
for me. I found
Number of servers you'll never know, that top1000 depends on admins
bothering to configure and send stats, for every server that does there's
probably 20 that don't - I know we don't.
As for articles, text groups, in past 22.5 hours (since last daily
maintenance run) we've received just over 21K articles, at about 51mb.
So grows about 1.5gb a month, roughly.
Thank you for the stats! (And for running Ausics. It's a nice idea to
provide an IP based access. Maybe I should do one just like that for my
country.)
Post by noel
far cry from years gone by where we'd see that and more in just one
day, but, some people like the eye candy bloatware of html on forums
now for some reason i'll never understand :)
It's understandable considering their skill level with the whole thing,
which is pretty skill level zero, nowadays even in computer science
departments. Sadly. :)
Marco Moock
2024-12-21 13:46:08 UTC
Permalink
Post by Salvador Mirzo
Thank you for the stats! (And for running Ausics. It's a nice idea
to provide an IP based access. Maybe I should do one just like that
for my country.)
Which includes the disadvantage that people from other countries are
excluded and abusers can just use proxies/TOR to connect.
--
kind regards
Marco

Send spam to ***@stinkedores.dorfdsl.de
Salvador Mirzo
2024-12-21 18:21:49 UTC
Permalink
Post by Marco Moock
Post by Salvador Mirzo
Thank you for the stats! (And for running Ausics. It's a nice idea
to provide an IP based access. Maybe I should do one just like that
for my country.)
Which includes the disadvantage that people from other countries are
excluded and abusers can just use proxies/TOR to connect.
That's right. It's not perfect.
noel
2024-12-23 11:55:59 UTC
Permalink
Post by Salvador Mirzo
Post by Marco Moock
Post by Salvador Mirzo
Thank you for the stats! (And for running Ausics. It's a nice idea
to provide an IP based access. Maybe I should do one just like that
for my country.)
Which includes the disadvantage that people from other countries are
excluded and abusers can just use proxies/TOR to connect.
That's right. It's not perfect.
there are exit node lists for TOR which allow blocking :)
and we've never had a proxy problem, again, because the proxy would be
within the country, the users traceable.
noel
2024-12-23 11:52:48 UTC
Permalink
Post by Marco Moock
Post by Salvador Mirzo
Thank you for the stats! (And for running Ausics. It's a nice idea to
provide an IP based access. Maybe I should do one just like that for
my country.)
Which includes the disadvantage that people from other countries are
excluded
how is that a disadvantage, it is by design and operated this way since
the 90's with great success and with no spammers or abusers, so its a win
win here.


(the country has ISP's who actually do enforce AUP's and since I've been
in the ISP game myself for nearly 30 years, I know the right people to
have enforcement action taken, not to mention laws under a few different
Crimes Acts that are enforced)
D
2024-12-21 14:42:58 UTC
Permalink
Post by Salvador Mirzo
Post by noel
Post by Salvador Mirzo
I'm looking for any notion of how busy USENET is right now, so the
number of servers and the number of articles posted by month is enough
for me. I found
Number of servers you'll never know, that top1000 depends on admins
bothering to configure and send stats, for every server that does there's
probably 20 that don't - I know we don't.
As for articles, text groups, in past 22.5 hours (since last daily
maintenance run) we've received just over 21K articles, at about 51mb.
So grows about 1.5gb a month, roughly.
Thank you for the stats! (And for running Ausics. It's a nice idea to
provide an IP based access. Maybe I should do one just like that for my
country.)
Post by noel
far cry from years gone by where we'd see that and more in just one
day, but, some people like the eye candy bloatware of html on forums
now for some reason i'll never understand :)
It's understandable considering their skill level with the whole thing,
which is pretty skill level zero, nowadays even in computer science
departments. Sadly. :)
perhaps they're too busy making perfect crop circles . . . proving that
aliens are people too . . . it's the nanny state of body, mind and soul
where denial is a river in egypt, . . and the streets are paved in gold
Salvador Mirzo
2024-12-21 17:23:40 UTC
Permalink
Post by noel
Post by Salvador Mirzo
I'm looking for any notion of how busy USENET is right now, so the
number of servers and the number of articles posted by month is enough
for me. I found
Number of servers you'll never know, that top1000 depends on admins
bothering to configure and send stats, for every server that does there's
probably 20 that don't - I know we don't.
As for articles, text groups, in past 22.5 hours (since last daily
maintenance run) we've received just over 21K articles, at about 51mb.
So grows about 1.5gb a month, roughly.
Wait---what do you mean "grows about 1.5gb a month"? Do you mean
traffic has been growing at about 1.5gb each month? IOW, it's growing
steadily lately?
D
2024-12-21 20:13:14 UTC
Permalink
Post by Salvador Mirzo
Post by noel
Post by Salvador Mirzo
I'm looking for any notion of how busy USENET is right now, so the
number of servers and the number of articles posted by month is enough
for me. I found
Number of servers you'll never know, that top1000 depends on admins
bothering to configure and send stats, for every server that does there's
probably 20 that don't - I know we don't.
As for articles, text groups, in past 22.5 hours (since last daily
maintenance run) we've received just over 21K articles, at about 51mb.
So grows about 1.5gb a month, roughly.
Wait---what do you mean "grows about 1.5gb a month"? Do you mean
traffic has been growing at about 1.5gb each month? IOW, it's growing
steadily lately?
including "admins bothering to configure and send stats":

(using Tor Browser 14.0.3)
https://top1000.anthologeek.net/participants.current.txt
Post by Salvador Mirzo
Total number of articles seen: 162479896
Total number of participants: 49
Total number of articles seen, after equalizing: 16822008
https://top1000.anthologeek.net/2024/11/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 238521959
Total number of participants: 75
Total number of articles seen, after equalizing: 28443388
https://top1000.anthologeek.net/2024/10/participants.txt
Total number of articles seen: 226027073
Total number of participants: 77
Total number of articles seen, after equalizing: 29867111
https://top1000.anthologeek.net/2024/09/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 218089654
Total number of participants: 77
Total number of articles seen, after equalizing: 28491785
https://top1000.anthologeek.net/2024/08/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 228442342
Total number of participants: 79
Total number of articles seen, after equalizing: 32054403
https://top1000.anthologeek.net/2024/07/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 164638782
Total number of participants: 79
Total number of articles seen, after equalizing: 31464573
https://top1000.anthologeek.net/2024/06/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 386248519
Total number of participants: 78
Total number of articles seen, after equalizing: 29012166
https://top1000.anthologeek.net/2024/05/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 882691916
Total number of participants: 77
Total number of articles seen, after equalizing: 30645294
https://top1000.anthologeek.net/2024/04/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 796871634
Total number of participants: 76
Total number of articles seen, after equalizing: 29777625
https://top1000.anthologeek.net/2024/03/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 525631221
Total number of participants: 76
Total number of articles seen, after equalizing: 32450187
https://top1000.anthologeek.net/2024/02/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 420511277
Total number of participants: 76
Total number of articles seen, after equalizing: 52683599
https://top1000.anthologeek.net/2024/01/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 437803273
Total number of participants: 75
Total number of articles seen, after equalizing: 66970429.
[end quoted excerpts]
D
2024-12-21 20:13:14 UTC
Permalink
Post by Salvador Mirzo
Post by noel
Post by Salvador Mirzo
I'm looking for any notion of how busy USENET is right now, so the
number of servers and the number of articles posted by month is enough
for me. I found
Number of servers you'll never know, that top1000 depends on admins
bothering to configure and send stats, for every server that does there's
probably 20 that don't - I know we don't.
As for articles, text groups, in past 22.5 hours (since last daily
maintenance run) we've received just over 21K articles, at about 51mb.
So grows about 1.5gb a month, roughly.
Wait---what do you mean "grows about 1.5gb a month"? Do you mean
traffic has been growing at about 1.5gb each month? IOW, it's growing
steadily lately?
p.s.

(using Tor Browser 14.0.3)
https://top1000.anthologeek.net/2023/12/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 507659455
Total number of participants: 79
Total number of articles seen, after equalizing: 88066199
https://top1000.anthologeek.net/2023/11/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 462129960
Total number of participants: 82
Total number of articles seen, after equalizing: 93834833
https://top1000.anthologeek.net/2023/10/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 410468912
Total number of participants: 84
Total number of articles seen, after equalizing: 95668567
https://top1000.anthologeek.net/2023/09/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 347086145
Total number of participants: 83
Total number of articles seen, after equalizing: 51199384.
https://top1000.anthologeek.net/2023/08/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 431897706
Total number of participants: 85
Total number of articles seen, after equalizing: 52195681
https://top1000.anthologeek.net/2023/07/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 374389646
Total number of participants: 85
Total number of articles seen, after equalizing: 51167214.
https://top1000.anthologeek.net/2023/06/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 420088591
Total number of participants: 84
Total number of articles seen, after equalizing: 46550820
https://top1000.anthologeek.net/2023/05/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 909905080
Total number of participants: 82
Total number of articles seen, after equalizing: 50015508
https://top1000.anthologeek.net/2023/04/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 7892654940
Total number of participants: 86
Total number of articles seen, after equalizing: 93990047
https://top1000.anthologeek.net/2023/03/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 9260940670
Total number of participants: 87
Total number of articles seen, after equalizing: 58428281
https://top1000.anthologeek.net/2023/02/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 7724776996
Total number of participants: 88
Total number of articles seen, after equalizing: 53438782
https://top1000.anthologeek.net/2023/01/participants.txt
Post by Salvador Mirzo
Total number of articles seen: 8387319943
Total number of participants: 85
Total number of articles seen, after equalizing: 55368074
[end quoted excerpts]

etc. (were googlegroups.com numbers ignored/not counted?)

anthologeek's monthly stats go back to august 2000:

https://top1000.anthologeek.net/2000/08/participants.txt
Post by Salvador Mirzo
Articles seen Pathname
========================================
16193858 news2.cistron.nl
5113488 news.fr.clara.net
30662773 newsfeed.freenet.de
824537 news.efnet.com
1494221 news1.radix.net
18910836 news.theplanet.net
noel
2024-12-23 11:53:12 UTC
Permalink
Post by Salvador Mirzo
Post by noel
Post by Salvador Mirzo
I'm looking for any notion of how busy USENET is right now, so the
number of servers and the number of articles posted by month is enough
for me. I found
Number of servers you'll never know, that top1000 depends on admins
bothering to configure and send stats, for every server that does
there's probably 20 that don't - I know we don't.
As for articles, text groups, in past 22.5 hours (since last daily
maintenance run) we've received just over 21K articles, at about 51mb.
So grows about 1.5gb a month, roughly.
Wait---what do you mean "grows about 1.5gb a month"? Do you mean
traffic has been growing at about 1.5gb each month? IOW, it's growing
steadily lately?
your storage :)
Julien ÉLIE
2024-12-22 11:28:36 UTC
Permalink
Hi Noel,
Post by noel
As for articles, text groups, in past 22.5 hours (since last daily
maintenance run) we've received just over 21K articles, at about 51mb.
Oh, interesting, I then wrongly thought there were less than this traffic.
Is your count of 21,000 articles a day after the run of spam and abuse
filters?

If I look at the statistics of Eternal-September, I only see about 3,000
articles a day in the Accepted column:
https://eternal-september.org/stats/index.html

I thought they had most of text groups. There is a huge difference (x7)
between the counts. Do you happen to know which hierarchies have most
of the 21,000 articles?

i2pn2 and nk.ca have both about 6,000 articles a day; pasdenom.info
about 4,000, so I am intrigued by your volume.
https://forum.i2pn2.org/
https://ns2.nk.ca/
https://pasdenom.info/usenet/index.html
--
Julien ÉLIE

« Passion is inversely proportional to the amount of real information
available. » (Benford's law)
Jesse Rehmer
2024-12-22 20:20:41 UTC
Permalink
On Dec 22, 2024 at 5:28:36 AM CST, "Julien ÉLIE"
Post by Julien ÉLIE
Hi Noel,
Post by noel
As for articles, text groups, in past 22.5 hours (since last daily
maintenance run) we've received just over 21K articles, at about 51mb.
Oh, interesting, I then wrongly thought there were less than this traffic.
Is your count of 21,000 articles a day after the run of spam and abuse
filters?
If I look at the statistics of Eternal-September, I only see about 3,000
https://eternal-september.org/stats/index.html
I thought they had most of text groups. There is a huge difference (x7)
between the counts. Do you happen to know which hierarchies have most
of the 21,000 articles?
i2pn2 and nk.ca have both about 6,000 articles a day; pasdenom.info
about 4,000, so I am intrigued by your volume.
https://forum.i2pn2.org/
https://ns2.nk.ca/
https://pasdenom.info/usenet/index.html
My stats show approximately 25MB per day, any typically around 8,000-9,000
articles per day.

After the Google Groups fiasco went away that seems to be the 'norm' for a
full text feed.

https://news.blueworldhosting.com

I would like to have more statistics on articles per hierarchy/group.
Julien ÉLIE
2024-12-22 21:55:47 UTC
Permalink
Hi Jesse,
Post by Jesse Rehmer
My stats show approximately 25MB per day, any typically around 8,000-9,000
articles per day.
https://news.blueworldhosting.com
Thanks for the link!

Incidentally, there's an outgoing feed declared as "cnfs" which does not
seem to do anything. Should it really be there?
Post by Jesse Rehmer
I would like to have more statistics on articles per hierarchy/group.
There's an analyze-traffic Perl script in the contrib directory which
counts the number of articles per newsgroup. I have not tested it, though.
I think people wanting such statistics write their own custom script
which parses their news spool (when using tradspool) or their history
file (with sm to obtain the articles) each month to have their monthly
statistics.
Some of them may be found on public repositories. The one I have in
mind is getflow which was written by Paolo Amoroso:
https://github.com/Aioe/getflow
--
Julien ÉLIE

« Contra factum non datur argumentum. »
Retro Guy
2024-12-23 16:22:29 UTC
Permalink
Post by Jesse Rehmer
On Dec 22, 2024 at 5:28:36 AM CST, "Julien ÉLIE"
Post by Julien ÉLIE
Hi Noel,
Post by noel
As for articles, text groups, in past 22.5 hours (since last daily
maintenance run) we've received just over 21K articles, at about 51mb.
Oh, interesting, I then wrongly thought there were less than this traffic.
Is your count of 21,000 articles a day after the run of spam and abuse
filters?
If I look at the statistics of Eternal-September, I only see about 3,000
https://eternal-september.org/stats/index.html
I thought they had most of text groups. There is a huge difference (x7)
between the counts. Do you happen to know which hierarchies have most
of the 21,000 articles?
i2pn2 and nk.ca have both about 6,000 articles a day; pasdenom.info
about 4,000, so I am intrigued by your volume.
https://forum.i2pn2.org/
https://ns2.nk.ca/
https://pasdenom.info/usenet/index.html
My stats show approximately 25MB per day, any typically around
8,000-9,000
articles per day.
After the Google Groups fiasco went away that seems to be the 'norm' for a
full text feed.
https://news.blueworldhosting.com
I would like to have more statistics on articles per hierarchy/group.
I do run a script on i2pn2.org that logs Newsgroups headers and counts
them.

It simply just increments the count for every time a duplicate is
encountered, so crossposts show up as an entry. I wanted it to check
some groups and don't really use it much, but I guess I could run it
more regularly if it's useful to some people.

The link below show the output for it running for about one month:

https://news.i2pn2.org/common/newsgroups_dat.txt

It gets logged into an array and the output above is from a dump of data
from that array.
--
Retro Guy
noel
2024-12-23 12:05:19 UTC
Permalink
Post by Julien ÉLIE
Hi Noel,
Post by noel
As for articles, text groups, in past 22.5 hours (since last daily
maintenance run) we've received just over 21K articles, at about 51mb.
Oh, interesting, I then wrongly thought there were less than this traffic.
Is your count of 21,000 articles a day after the run of spam and abuse
filters?
yep, the rejects daily for last run was about 2K, long term is skewed due
to the google trash, but it becomes clearer with more months passing
since they went away.
Post by Julien ÉLIE
If I look at the statistics of Eternal-September, I only see about 3,000
https://eternal-september.org/stats/index.html
I thought they had most of text groups. There is a huge difference (x7)
between the counts. Do you happen to know which hierarchies have most
of the 21,000 articles?
Not off hand, dnews doesnt give that sort of reporting, only top groups.
Post by Julien ÉLIE
i2pn2 and nk.ca have both about 6,000 articles a day; pasdenom.info
about 4,000, so I am intrigued by your volume.
https://forum.i2pn2.org/
https://ns2.nk.ca/ https://pasdenom.info/usenet/index.html
I've had a quick look at a few servers a couple months ago when
cybershock was doing their latest update, a number of those servers do
not carry all the aus.* groups, some apparently none at all (out of the
groups I tested), if that happens for aus.* its more than likely
happening in other hierachies.
noel
2024-12-24 01:27:29 UTC
Permalink
Post by noel
Not off hand, dnews doesnt give that sort of reporting, only top groups.
Post by Julien ÉLIE
i2pn2 and nk.ca have both about 6,000 articles a day; pasdenom.info
about 4,000, so I am intrigued by your volume.
https://forum.i2pn2.org/
https://ns2.nk.ca/ https://pasdenom.info/usenet/index.html
I've had a quick look at a few servers a couple months ago when
cybershock was doing their latest update, a number of those servers do
not carry all the aus.* groups, some apparently none at all (out of the
groups I tested), if that happens for aus.* its more than likely
happening in other hierachies.
I've had a look in the configs, the numbers here may also be skewed by us
taking feed from "lore", one of my team has added a comment saying not to
be exported, sent her an email since I'm on holidays asking why as I'm
curious, but earlier comments still stand, by your own stats there is a
wild variance of article numbers by servers.

Perhaps they don't want some hierachies, or blocked some for legal
reasons, we have to block certain groups when requested by ASIC or the
AFP (that list is currently only 7 groups anyway)

Happy Holidays

Loading...