Peering implementations

[immibis]

What format do NNTP peers usually take?

I see the "suck" tool can download import articles from any
public-access server without special negotiation. Does it put excess
load on the upstream server?

Most people are using innfeed, right? innfeed peering is either IHAVE
(chatty) or CHECK/TAKETHIS (streaming), is that right? Is there any
reason to use IHAVE except for compatibility with older servers?

How's the connection managed? In other streaming protocols I'm more
familiar with (Redis, Apache Kafka) a receiving client would connect to
a server that has messages, state its last synchronization point and
then downloads messages from that point on. If the connection is
interrupted, the messages still arrive upstream and get stored until
they're pulled by the receiver.
NNTP streaming doesn't have that feature, and I think innfeed is
designed so upstream servers try to push articles downstream, rather
than downstream ones pulling them, right? And failures to a separate
spool for re-processing?

Is there really no better way to authorize a connection than checking
the other side's IP address?

Besides UUCP, is anyone using anything exotic?

(I should set up a server with a Kafka database and convince people to
peer directly to Kafka. Sounds !!fun!!)

[Marco Moock]

Post by immibis
What format do NNTP peers usually take?
I see the "suck" tool can download import articles from any
public-access server without special negotiation.

They use peering and not the reader mode that suck will use.

Post by immibis
Does it put excess load on the upstream server?

Depends on the amount of article you load.

Post by immibis
How's the connection managed?

Both sites need to configure their NNTP server to have outgoing
articles and incoming.

Post by immibis
In other streaming protocols I'm more
familiar with (Redis, Apache Kafka) a receiving client would connect
to a server that has messages, state its last synchronization point
and then downloads messages from that point on. If the connection is
interrupted, the messages still arrive upstream and get stored until
they're pulled by the receiver.

IIRC INN can spool messages if a server is unreachable (the statistics
sometimes show that). But for servers that are not always online (most
time), UUCP is a better protocol.

Post by immibis
Is there really no better way to authorize a connection than checking
the other side's IP address?

Using a TLS certificate should be technically possible, but I don't
know which servers support that.

[Stefan Ram]

Post by immibis
I see the "suck" tool can download import articles from any
public-access server without special negotiation. Does it put excess
load on the upstream server?

Without knowing further details, a first superficial examination
of this question would lead me to think that it does not seem
unfair to use such a program, because the server is free to
refuse anything that would expose it to unwanted overload.

[Grant Taylor]

Post by immibis
What format do NNTP peers usually take?

Peers usually push articles to each other.

Readers usually poll / pull articles from the server(s).

Post by immibis
I see the "suck" tool can download import articles from any
public-access server without special negotiation. Does it put excess
load on the upstream server?

It depends.

I would never want to use suck for a full Usenet feed for anything other
than academic purposes.

Post by immibis
Most people are using innfeed, right? innfeed peering is either IHAVE
(chatty) or CHECK/TAKETHIS (streaming), is that right? Is there any
reason to use IHAVE except for compatibility with older servers?

I would have to refer to RFCs to know for sure, but these seem to be
ways to push articles to peers. Which is used when requires brushing up
on some RFCs.

I believe that innfeed is just one of the ways that INN supports to feed
articles to a configured peer.

Post by immibis
How's the connection managed? In other streaming protocols I'm more
familiar with (Redis, Apache Kafka) a receiving client would connect to
a server that has messages, state its last synchronization point and
then downloads messages from that point on.

My understanding is that INN (which uses innfeed) has knowledge of
articles that are queued to be sent to a peer and which queued articles
have been sent.

With this in mind, INN simply sends articles for desired newsgroups /
distributions to wanting peers henceforth.

Peers don't have a way to say I want you to send me article <such and
such>. That's the domain of the reader to pull articles.

Post by immibis
If the connection is interrupted, the messages still arrive upstream
and get stored until they're pulled by the receiver.
NNTP streaming doesn't have that feature, and I think innfeed is
designed so upstream servers try to push articles downstream, rather
than downstream ones pulling them, right?

My understanding of the mechanics may be flawed.

Post by immibis
And failures to a separate spool for re-processing?

In a manner of speaking, sort of.

INN will periodically try to establish communications and will send (or
at least offer) spooled articles once communications is established.

Post by immibis
Is there really no better way to authorize a connection than checking
the other side's IP address?

What is "better"? IP is quite convenient and works quite well between
static (or rarely changing) IPs.

You could use something like IPsec transport mode, or a full fledged VPN
to have much stronger cryptographic validation about the remote system.
But is such worth it for a news server?

Post by immibis
Besides UUCP, is anyone using anything exotic?

I believe that some have TLS encryption in play.

I believe a very small number are using NNCP (?)

I would not be shocked to learn about IPsec / VPN being used.

Post by immibis
(I should set up a server with a Kafka database and convince people to
peer directly to Kafka. Sounds !!fun!!)

Why?

IMHO that wouldn't be worth it.

What reason do you have to upset the apple cart? -- This is one of the
reasons that I'm not interested in NNCP.

I'd be more inclined to use standard NNTP on a standard news server and
rely on the kernel to add IPsec transport mode encryption +
authentication to the traffic. News software doesn't have to be
involved and can rely on the kernel.

[Julien ÉLIE]

Hi immibis,

Post by immibis
Is there really no better way to authorize a connection than checking
the other side's IP address?

FWIW, innd also supports identifying a peer with the Ident protocol (RFC
1413) or with a password.
I doubt they are really used by people nowadays. Besides, these data
are not encrypted.

See the identd and password parameters in incoming.conf:
https://www.eyrie.org/~eagle/software/inn/docs/incoming.conf.html

peer bob {
hostname: "1.2.3.4"
password: "therealbob"
}


You may use that as an additional "proof" (because that's still
insecure) that the peer at 1.2.3.4 is the expected one.

Note that you cannot use "*" as hostname, and then parameter several
peers matching any IP but with several different passwords because the
first matching entry for hostname in incoming.conf, read from the last
one to the first one in the file, will be used. If the conditions like
max-connections, identd and password do not correspond, then access is
denied.

[Julien ÉLIE]

Hi immibis,

Post by immibis
Is there really no better way to authorize a connection than
checking the other side's IP address?

FWIW, innd also supports identifying a peer with the Ident protocol (RFC
1413) or with a password.
I doubt they are really used by people nowadays. Besides, these data
are not encrypted.

See the identd and password parameters in incoming.conf:
https://www.eyrie.org/~eagle/software/inn/docs/incoming.conf.html

peer bob {
hostname: "1.2.3.4"
password: "therealbob"
}


You may use that as an additional "proof" (because that's still
insecure) that the peer at 1.2.3.4 is the expected one.

Note that you cannot use "*" as hostname, and then parameter several
peers matching any IP ("*") but with several different passwords because
the first matching entry for hostname in incoming.conf will be used for
the upcoming connection. If the conditions like max-connections, identd
and password do not correspond, then access is denied.