[isf-wifidog] WiFiDog Roadmap, first draft

[Benoit Grégoire]

I everyone, I am back ;)  I am extremely happy about the work that was done in 
my absence, congratulations!  

That being said (yes, there is always a "but..." with me),   I am quite 
surprised by some of what I heard in this thread.  The auth server clearly 
has an image problem, even (or especially) within ISF.  I'll try to address 
this a little bit, and try to at least start a Roadmap.

Many of the requests or complaints I heard are already addressed by what the 
auth server already does, where it's already going or would only require tiny 
changes to be in the mainline distribution.  This very long message is meant 
to give a little perspective about where WiFiDog is, and where it may be 
going.

First there is the gateway.  It is designed to be simple and quite dumb.  The 
only UI on it is to get it's status, and display a message in case of 
abnormal conditions (auth server down, internet down, etc.).  It is not 
meant, and probably never will run without some kind of remote auth server.  
In the course of it's development, we've met with various bugs and problems, 
but our initial design has proven to stand the test of time in the real 
world.  See http://www.ilesansfil.org/wiki/WiFiDog/FlowDiagram.  The gateway 
roadmap for now is:
-Traffic shaping.  Four different shaping related issues will eventually have 
to be adressed:
	-Interface shaping:  Avoiding latency spikes caused by concurrent users due 
to buffer contention in the DSL or cable modem.  This could be solved by 
existing solutions like WonderShaper, but there needs to be a better way to 
set the available bandwidth.  Ideally it should be measured by a script, as 
DSL lines can vary very significantly from the advertised bandwidth.
	-Fair bandwidth allocation between concurrent users. (A single user opening 
ten connections should have the same bandwidth as his neighbour opening a 
single connection)
	-Monthly bandwith limiting by hotspot.  (Must include use outside the captive 
portal)
	-Slow down users once they have used excessive bandwidth at a single hotspot.
-WPA support:  This is a long term project.  As drivers mature, it becomes 
(well, probably) workable to optionally skip the login page by authenticating 
using WPA.  However, this will need strong abstraction if we don't want it 
confined to the WRT54G.  Also, it must use conditional compilation, as just 
this code will probably be larger than the whole of the of the current 
wifidog gateway codebase.  Note that the goal of this isn't to eliminate the 
captive portal, just to skip login page and gain "wireline" encryption.

Now the auth server.  It has (and always had) two major functions for ISF: 
peer-hotspot content delivery and network management (authentication, 
statistics, network monitoring, etc.).  A number of complaints were expressed 
in the last two months:

* It requires PHP5:  Well, PHP5 has been released for a year now and is 
officially or semi-officially part of every major distribution.  Doing what 
we are doing with PHP4 would be complicated, much more brittle and ultimately 
a non-issue.  Yes, right now few hosting providers offer PHP5 services, and 
some groups do not want or cannot run their own auth server.  But the problem 
will solve itself over time, and I don't see why anyone would put in the 
several weeks (minimum) to reach feature parity with a PHP4 fork.  Any such 
fork wouldn't last all that long unless it fills a VERY specific niche.

* It only works with PostgreSQL:  Ok, first I need to vent a little.  Ever 
since we moved from MySql to Postgres (mostly because Postgres is the 
de-facto GIS database), we had people request a MySQL version.  For this to 
remain possible, we can't use most of postgres's advanced features.  For the 
same reason, very large efforts were made to stick strictly to SQL99 
features.  The database abstraction is there.  All this was done at the cost 
of much more work, and a more brittle data model.  Not much needs to be done 
to make it work.  Currently the initial schema is auto-generated from a 
working install to avoid errors (this is what update_sql_for_cvs does), as 
many postgres attributes as possible are stripped off with dump parameters.  
All someone needs to use it with mysql is either a few minutes to port it to 
MySql everytime it changes (mostly adding InnoDB table types so foreign keys 
will work, and replace a few types MySQL does not directly support).  The 
best however is for someone to write a few lines 
of regular expressions to do it automatically.  This could then be done for 
every commit so everyone can benefit.  

Unfortunately, so far all this work didn't pay off.  People complain, no one 
uses it and we are stuck with the lowest common denominator.  If someone 
actually steps forward to maintain MySQL support, I'll personally help him 
fix all the small problems that will crop up  (by now dozens of queries are 
untested with mysql).  If no one steps up in the near future, I'll propose 
that we completely give up on MySQL to be able to use Postgres more fully.  
To summarise, what must be done to make MySQL fully supported again:

-Write a script to convert the schema to mysql automatically.
-Make sure initial data works with both databases.
-Correct the different queries, making sure that they remain SQL99 compliant.  
When that is truly impossible, use the constants in the config file to send a 
different query to mysql.

* The auth server is getting too big:  “too” big is subjective, but it's true 
that it's getting bigger (just passed 10000 SLOC).  Some have suggested we 
start over and make a core auth server with a modular architecture.  I think 
that's a very bad idea.  Even with the current auth server as a prototype, 
starting over and reaching feature parity would be WAY more work than 
progressively refactoring, as we are quite far along already.  The auth 
server is ALREADY quite modular, and it's getting more modular by the day.    
Already we have:
-Pluggable authentication modules (Local DB and RADIUS already working, 
Pear-Auth trivial to add once PHP5 compliant)
-Pluggable Content delivery (See 
http://www.ilesansfil.org/wiki/WiFiDog/ContentDistributionSystem for early 
documentation on it's features). It's easy to add new types, the classes are 
even self-enumerating.  Just add a php class file in class/content/.  There 
would be much to discuss about the internal design of this subsystem which I 
am very proud of, but that will have to wait.
-Most external dependencies can already be disabled without preventing the 
server to work.
Now some people seem to think that it would be a good idea to just release a 
so called "core" server.  Frankly, I don't get it.  Such a core server would 
need to at least support language translations, a database (and schema 
versioning), and some form of local content.  As modularization continues to 
improve, it will be possible to distribute simple plugins.  But managing 
adding database tables  and language translations specific to an external 
plugin would be a nightmare both in logistics and for stability.  Such 
plugins would need their own versioning for each table.  That would 
definitely not make the system simpler.  We should distribute most of the 
contributed modules as part of the main distribution.  They do not hurt 
performance (this is all interpreted code...).  What we SHOULD do however is 
make it a rule of inclusion that any module detects the absence of any of 
their dependencies at runtime and deal with the condition gracefully. That 
way we can keep a single translation source file, and unified documentation.

* There is no splash only auth server.  This is an important feature, but it's 
very easy to add to the current server.  In fact I intended to do just that 
to pre-empt this request, but I haven't had time yet.  In any case there is 
no reasonable case for a separate splash server that wouldn't require a 
database.  A much better use of developer time would be to work on SqlLite 
support (especially since it's built into PHP5).

Roadmap:

What has recently been added to WiFiDog:
-Better object model.  There are now almost completely abstract objects for 
most important concepts (Network, a Node, User, authentication sources 
(Authenticator), Content, Content Group, etc.
-Because of the much better object model, we are starting to have much better 
internal documentation.  Hopefully this could become readable enough to serve 
as an architecture document for the time being.
-Database schema versioning and automatic upgrade of an existing auth server 
installation (schema_validate.php).  This allows many database changes to be 
made between release, and makes the migration process automatic.  This check 
is triggered by accessing the root page of the aut server.
-"Formless" form handling.  Every class that has an interface should implement 
a few standard methods to allow it to display and process it's user and admin 
UI without knowing anything about the URL of the form that uses it.  This 
allows you to re-use an object's UI in any context, and allows administration 
of most objects with a single form.
-Moved away from Smarty templates for most uses.  Not that Smarty isn't great, 
but it's not terribly well suited to our problem.  The UI is now mostly tied 
to objects instead of pages and we now have a CSS only layout.  Had we 
retained Smarty, the number of templates would have exploded, each template 
wouldn't contain any layout anyway, and we would have had to retrieve the 
html generated by Smarty because it has to be passed back to the parent 
object or the calling method.  Because of this Smarty increased  complexity 
of most of the code while no longer improving clarity.  Note that Smarty is 
still supported, just not used very much.


What is currently in progress:
-Priority:  Finish transitioning legacy code to the new design, which will 
massively drop the barrier to entry both for administrators and developers.  
Because we (and other people) have wifidog auth servers in production, not 
all features have been ported to the new design.  Two specific areas are 
currently a messy mix of legacy code and new elements.  
1- Statistics.  Many of the underlying functions of the reports are to be 
deprecated (From now on, getters should never return complete arrays of RAW 
sql if they can return objects.  If they can't, the query probably belongs 
directly in the report code.
2- Node administration.  It's currently spread out over different pages, some 
of which have functionality related to the local content directories that no 
longuer has any effect (some changes are incomplete), yet the documentation 
has not yet been updated.
-Network abstraction, and then multiple network support.
-Auto-detection of enabled features depending on installed dependencies.
-On line documentation.  Some subsystems like the Content delivery system and 
parts of the network management interface are not self explanatory and should 
have popup help in the web interface.

There will probably be official releases before this point.  However, this is 
probably the 1.0 feature set.

Possible future features (In no specific order, only ISF features listed, we 
need thos of other groups...):  

-A design philosophy document (it should be a wiki page)
-More elegant handling of network administrators.
-Explicit disclaimer (terms of service) support.  This is essential for ISF, 
and is the ONLY thing many groups want.
-Allow at hotspot to be defined as splash only (no login) if the network 
allows it.
-Eliminate config files, everything should be doable from the web interface 
and stored in the database.
-User profiles (Allowing every user to opt in to have a profile page, allowing 
them to change their username, etc.
-Search engine for some reports.
-Per user MAC address white-listing (If enabled by the network, probably under 
a different policy than full token authentication).  This may require some 
changes on the gateway.
-Allow the auth server to lookup nearby hotspots for an active session that 
matches the current MAC address.  Combined with a good addressing plan, this 
will allow correct operation in hotspots in overlapping coverage zones.
-A PHP gateway simulator, to act as a development and debuging tool.
-Restore tho old local_content functionality as a Content type.  This will 
allow people to include web page fragments and have the relative image links 
work. While this is rarely needed (it duplicates the work of several of the 
content types), it can be usefull in some circumstances.

General design:

I think the wifidog auth server should be the swiss army knife of auth 
servers. And I believe this can be done without being bloated if we execute 
it well.  But we urgently need to finish cleaning up the code and write 
documentation.  We finally see massive interest from other groups to 
contribute.  Spending time to make their job easier and hold their hand is a 
priority, even if it slows development down in the short term.  Even if 
nobody contributes, it will make OUR job much easier in the long run.

Anyway, this document is what I came up with with a bunch of short writing 
breaks on airplanes, in airports and during the very few free times available 
at FISL 6.0.  It was not revised, and I have the feeling I will reply to 
myself as soon as I post it.  I hope that this will form a good basis for 
discussion and that I didn't offend anyone (I didn't have Internet access 
while writing, and I hope I didn't miss anyone's point).  Hopefully it will 
at least give other groups a slightly clearer idea of where we may be going.  
I'd really like to get their feedback and feature requests.  There will be an 
ISF developer meeting very soon where we can hopefully tie this together a 
little more. 

Good night,

-- 
Benoit Grégoire, http://benoitg.coeus.ca/
-------------- section suivante --------------
Une pièce jointe non texte a été nettoyée...
Nom: non disponible
Type: application/pgp-signature
Taille: 189 octets
Desc: non disponible
Url: http://listes.ilesansfil.org/pipermail/wifidog/attachments/20050606/838864f9/attachment.pgp