[isf-wifidog] Ok, the real message: RADIUS, MySQL, Postgres, reimplementations & the Sherbrooke meeting and other things

[Benoit Grégoire]

On May 26, 2009, Marc Blanchet wrote:
> wlanmac a écrit :
> > The question becomes, where should it be optional?
> >
> > Optional in the way it is now where it is used by the portal to be
> > compatible on the roaming side,
> >
> > AP/WiFiDog ---> <wifidog protocol> ---> WiFiDog Portal ---> RADIUS
> >
> > or optional for access provisioning to the router, for device / access
> > controller compatibility?
> >
> > AP/NAS ---> <RADIUS> ---> <something?> ---> WiFiDog Portal
> >
> > (Note: "NAS" for Network Access Server is a more generic term for access
> > controller in RADIUS documentation).
> >
> > A concept I have been thinking of would be something along the lines of
> > what Marc suggested. A proxy to map the AAA features of a NAS with the
> > WiFiDog HTTP protocol.
> >
> > Such that you might have:
> >
> > AP/NAS/Proxy ---> <wifidog> ---> WiFiDog Portal (to support old-school
> > sites)
> >
> > AP/NAS ---> <RADIUS> ---> Proxy/WiFiDog Portal (to support standard
> > access controllers)
> >
> > AP/NAS ---> <RADIUS> ---> <FreeRADIUS/SuperProxy> ---> Multiple WiFiDog
> > Portals based on Realm (the future)
> >
> > It's just an idea... but, could be a way of achieving the best of all
> > worlds, RADIUS-wise.
>
> you were just 5 minutes faster than me to write almost exact same
> drawings... a few minor edits below:
>
> this is my view of v2:
>
> "v1" AP -> wifidog-http -> v2 wifidog portal -> radius backend
> "v2" AP ->              radius               -> radius backend

You just essentially described CoovaChilli, but even with Coova there is still 
a piece of software to run on the leaf node, and a piece of software other 
than the RADIUS server to run on the server for full full functionnality.  But 
if this is what you want, continuing with wifidog is just plain silly (assuming 
you use it)  Coova is really good at what it does, and I believe it's main 
developer has exactly the same RADIUS-centric vision as you do.

It is funny however to see people suggest using RADIUS would make everything 
simpler to understand and implement.  Unless someone spends most of his 
workdays working with RADIUS and writing FreeRADIUS plugins, I find that well 
beyond hard to believe.  The mere LIST of distinct RFCs specific to RADIUS 
(http://en.wikipedia.org/wiki/RADIUS#RFCs) is half as long as the current 
wifidog protocol documentation: 
http://dev.wifidog.org/wiki/doc/developer/WiFiDogProtocol_V1

"Using RADIUS" is just about as precise in the wifidog project context as 
"Using http".   There is the RADIUS UDP transport protocol then there are the 
various extensions, the RADIUS servers themselves, and all the modules and 
backends to get them to help you manage networks.

But when people say "let's use RADIUS" they probably have a much more precise 
meaning in mind. I think the reasoning goes like this:  
1- RADIUS is designed to manage users
2- RADIUS databases are frequently replicated
3- RADIUS has provisions for roaming between servers
4- Several switches and access points have built-in RADIUS support
5- RADIUS is used to manage large networks, including wireless ones.
Therefore, RADIUS probably does everything wifidog does, and does it better.

The reasoning is logical.  But indicates either a misunderstanding of what 
wifidog is, or of what RADIUS provides.

1- User login handling is a relatively minor aspect of WiFiDog.  It's central 
to RADIUS.  Minor for wifidog but with a rather unique twist: the focus 
managing hundreds of thousand users that are allowed to self-subscribe and 
freely use the network, and then manage abuse of any single user without human 
intervention.  Furthermore, the system has to track displaying content taking 
while taking into account which content the user has previously seen and 
where.  Basically manage users as a big set, not individually (for the longest 
time, there wasn't even a facility to delete them, because in general it's 
undesirable to do so).   If user's can't self-subscribe WiFiDog just plain 
doesn't care where the username-password pairs come from.  RADIUS, LDAP, 
whatever.  But even if we did rely entirely on a mandatory RADIUS backend, we 
would have had to write a user management backend for said RADIUS server to 
provide all these services.  Relying on a RADIUS server and the use it as a 
dumb user/password store just doesn't make sense.
2- Quite true.  However (unlike LDAP) database replication isn't part of the 
scope of the RADIUS specs.  A typical radius server with a MySQL backend will 
use the mysql replication facilities to replicate.  A radius server with a 
LDAP backend will use LDAP facilities to replicate.  A RADIUS server with a 
file backend will use rsync.  You get the idea.  In any event, one cannot 
replicate two radius servers running different RADIUS software together simply 
because they are RADIUS servers.
3- It does, but it doesn't make figuring out the realm automatically (which is 
the real UI problem) any simpler or harder.  That being said, using RADIUS for 
inter-auth server communication would make perfect sense, if it didn't 
suddenly add the dependency of an operational freeradius server, and of 
writing a freeradius module to authenticate against the wifidog database 
(although this would potentially make WPA encryption available at the same 
time)
4- They do, but that doesn't make them support dynamic walled garden, 
equitable bandwidth share between users, centralizing gateway configuration and 
firewall settings automatic or even any easier.  If they did, there would 
hardly be any need for CoovaChili or the wifidog gateway either.  
5- That's a valid point.  I personally believe the techniques applied by the 
ISPs (which are the main force behind RADIUS) are frequently insufficient and 
conter-productive.  But that is very much open for debate.

The RADIUS protocol is a UDP protocol designed to transmit network credentials 
and usage statistics between an authentication source (typically a database of 
some kind) and a network access controler (a modem, a switch, these days a wifi 
router).  As such it's applicability may seem obvious.  Various extentions 
define additional attributes (but have very uneven level of support between 
implementations). 

------- The gateway ------

Indeed at the genesis of the wifidog project (End of 2003),  Using the RADIUS 
protocol for gw-auth server communication was considered, and rejected.  The 
protocol is simple (more so than http in many ways).  But It clashed with 
points 1 and 4 of the original gateway requirements which were:

1- Be embeddable in even the smallest systems.  Indeed we got the gateway down 
to 16kB.
2- Work behind any proxy or firewall, and be completely unaffected by NAT
3- Offload as much intelligence and processing to the auth server.
4- Only require a web server to implement an auth server.
5- No dependencies on crypto libs if at all possible.

Indeed the several alternate auth server implementations for specific needs 
have been written for the WiFiDog 1.0 protocol.

While this is not the focus of the current sprints by the Quebec wireless 
groups that agreed to put their efforts in common a few weeks ago, there is a 
clear need to have a WiFiDog 2.0 protocol, to support the following features:

1- Have all the nodes run identical images, INCLUDING configuration
2- Dynamic walled garden (For OpenID support, mesh backhaul nodes 
whitelisting, allowing web mashups on the login page)
3- Allow storing static and dynamic node-specific firewall rules on the auth 
server if for no other reason than to finally get rid of the TrustedMacList 
gateway config parameter)
4- Dynamic bandwidth management (as pecentage of available bandwidth, not just 
an absolute number of bytes/sec for a class of user)
5- Allow the auth server to tell the gateway when to contact it next.

This COULD all be implemented as RADIUS protocol extensions, but once again 
RADIUS wouldn't make it any simpler. 

------------ The auth server ---------------

Now what was the original focus of the auth server?
1- Allow people to self-subscribe, and self-verify their email address
2- Present location specific content to the user, using content stored OUTSIDE 
the auth server as much as possible.
3- Be fully multilingual
4- Manage abuse

Since then it's grown in scope, and we didn't fill these four original goals to 
anyone's complete satisfaction.  Explaining exactly what we were trying to 
build in early 2004, and the successes and failures we had could fill a book.  
Yet it can be illustrated simply with today's technology:

We were trying to build a fully location and view history aware Liferay.

But even in 2009 we haven't yet consolidated on a standardized, language 
neutral portlet specifications with widely available open server 
implementations. But it is (excruciatingly) slowly getting there.

There was a decision in Sherbrooke to write a specification and API to 
completely decouple the content serving from the auth server.  This will be 
very, very interesting work.  It will also be very, very difficult work to make 
it a reality.  I've done standard writing work before.  Avoiding groupthink 
and allowing implementations that are so lowest common denominator as to make 
using the standard pointless, while making it easy to implement and popular is 
never easy.  

It will not make the auth server simpler either (on the contrary).  But it 
must be done if we are to have any kind of meaningful l impact on helping 
making delivering truly location specific content to end users a reality.  It's 
the only MAJOR technical undertaking agreed on in Sherbrooke, but it's an 
important one to the future of the project.

-----  PostGres vs MySql   ------

As for PostGres vs MySql, I won't get into that debate again.  It's pointless.

I just give some usefull information for anyone who intends to revisit the 
issue:
-MySql used to be supported, but support for it was dropped because we 
couldn't find ANY developer to maintain it.
- http://bugs.mysql.com/bug.php?id=2364 would make automatic schema upgrade of 
live servers like we have now excruciatingly slow once you have a fairly large 
connetion history.  Not to mention that database modification statements are 
not transactional in MySql, so we'd have to do a lot more testing, so we 
couldn't do schema updates as often as we do now.
- The inability to determine which row is kept in in GROUP BY statements in 
mysql will require architecting the statistics subsystem very differently if 
one wants to maintain comparable performance. 
- If I were to start from scratch today, I'd probably investigate the 
feasibility of basing the authserver on something like CouchDB  (real support 
for asynchronous multi-master replication, map-reduce, keep real ACID support 
at the cost of being non-relational).  

In any event I'm very much open  to a project to re-add MySql support IF it 
brings new developers along (i know it WILL bring new users along, but it's a 
disservice to everyone if we can't support them like last time we supported 
MySql).  

Teaching both PostGres ans MySql classes and doing data-mining on both, I must 
say that if someone manages to make all the database queries MySql and 
Postgres compliant while keeping dynamic schema upgrade and near realtime 
stats on a 1gb + wifidog dataset I'll be quite favorably impressed ;)  But it 
IS doable, with some compromises.

------ Why not start over ------

I've said most of this while in Sherbrooke, and a consensus was reached. 

Yes, the wifidog auth server is very big.

Writing a basic auth server that displays a different web age to every user is 
a < 1week project in a modern dynamic language (Python, Ruby and even PHP) for 
a competent programmer.  In fact that's the time it took one person to write 
the original implementation of the auth server.

Writing a generic auth server that is easy to install, and meets the needs of 
disparate communities and is structured to accept outside contributions is 
something completely different.

Most of the complexity in wifidog isn't in user management, or gateway 
interaction, or any of the things I see people debate when they say that it 
would be easier to start from scratch.  Most of the complexity is in:

-Location-aware and history-aware content delivery and management
-User permissions
-Web based installer, dependency install and automatic schema upgrade
-The statistics subsystem
-Pervasive multilingual support
-Making everything generic enough that we don't end up with spaghetti code.

Off course, wifidog support many, many other features, but code-wise they are 
comparably very small.

If you have specific needs, ask how we think they could be integrated in wifidog 
without causing problems to anyone else.  If you can't do it, by all means do 
roll your own.  With any luck, we'll have some good ideas to steal from ;)  

------ What will happen in the next few months -----

In the meantime, I expect very good things to come from the meeting in 
Sherbrooke.  I've seen open minded people with actual resources to commit who 
agreed on cooperation and realistic and achievable goals.  The patches are 
already rolling in.  

I'll do my best to help the new contributors, and integrate the patches.  But 
very soon, other people will do most of that job as well.

If I become "just another contributor" I'll consider that a major victory for 
the project.  


-- 
Benoit Grégoire



-------------- section suivante --------------
Une pièce jointe HTML a été nettoyée...
URL: http://listes.ilesansfil.org/pipermail/wifidog/attachments/20090526/39c36c23/attachment-0001.htm