Nav-dev

nav-dev@lister.sikt.no

96 discussions

Unifying the source tree
by Morten Brekkevold 21 Jul '10

21 Jul '10

Hi all, Just quick heads-up about what I'm spending my idle time doing at EuroPython 2010. I've begun work on the blueprint https://blueprints.launchpad.net/nav/+spec/unified-source-tree , which was proposed by Kristian about a year ago. The blueprint is spot-on: As we've managed to reduce the number of programming languages involved in NAV, and as we are introducing more testing, having the subsystem structure in the the source code tree sucks. Basically, all the code that's left in the subsystem directory is Python code, and most of it is installed into the same package hierarchy. This means it makes no sense to split the source up into multiple "sub-project" directories. So here's what I've done so far: I've taken everything out of the subsystem directory and dispersed it into a new layout (as described by the full specification at http://metanav.uninett.no/devel:blueprints:unified_source ). Also, we've been porting more and more of the web-based subsystems to Django. mod_python is officially dead, so we should really get up to speed on moving the legacy apps away from the native mod_python interface. A first baby step towards cleaning up some things here has been done on my branch: Only static media files are installed in the Apache DocumentRoot (@webroot@) now. All python code goes in the library. All the .htaccess files have been deleted. A top-level etc/apache/apache.conf contains the default vhost config for NAV, which defaults to a Django handler (and using our existing mod_python headerparserhandler). Any exceptions to this rule can be configured by dropping a file in etc/apache/subsystems which uses <Location> to configure the exception. Exceptions from the default have been added for all native mod_python-interfacing apps and for static media. Now, a few words of warning: - This has not been tested yet. It's just a bunch of rampant changes to the directory structure, with some added config changes. - I found a several pieces of old documentation that was patently wrong or otherwise outdated. Rather than attempt at updating it, I deleted it. - We're still stabilizing 3.6, some of which involves adding new files. Merging series/3.6.x on to this branch when new files have appeared will require manual intervention in the merge working copy before committing. Files will not be magically added to the correct subdirectories on merge, since mercurial doesn't version directories, only files. - If you're working on a feature branch, please clone it and do a test-merge of my branch to see how it affects your changes. Until I'm satisfied that my changes work, changes should only be merged onto my branch, not the other way around. - Not sure I'll have time to test the changes properly before I get back from EuroPython and finish my three weeks of vacation. So, finally: The branch is located here: http://metanav.uninett.no/hg/features/unified-source-tree/ So much for being "quick" ... Cheers from Birmingham! -- mvh Morten Brekkevold UNINETT

1 0

Django 1.2 CSRF verification failed
by Morten Brekkevold 13 Jul '10

13 Jul '10

I just got an e-mail from someone trying to install NAV with Django 1.2. He was unable to log in to NAV, as the login page would produce a non-descriptive error about a CSRF token not being present. The user solved his Django 1.2 problem by following the steps in this blog post: http://josh.gourneau.com/blog/2010/02/17/django-12-csrf-verification-failed/ The Django documentation for Cross Site Request Forgery protection is here: http://docs.djangoproject.com/en/dev/ref/contrib/csrf/#how-to-use-it I'm not quite sure why this affects NAV; I'm not aware that we have employed any of this before, and we don't use the Django admin interface. Any suggestions to how we could solve this to work for Django versions 1.0 through 1.2? -- Morten Brekkevold UNINETT

2 1

Summer plans, 3.7 targets and more
by Vidar Faltinsen 24 Jun '10

24 Jun '10

Hi all, Here are some notes from the NAV meeting on Friday June 18 regarding summer plans and what we target for NAV 3.7. (Summer will be a lot about bug fixing 3.6 also). NAV 3.7 ======== Date for 3.7 beta: Oct 1: Blueprints included (see https://blueprints.launchpad.net/nav) Morten ------ * getBoksMacs to ipdevpoll * qbridge mib * lldp support * autoinsertion of type * OSM integration (room table schema, seeddb additions) Fredrik ------- * threshold interface * AD support in ldap Magnus ------ * sudo Thomas ------ * Automated builds and tests of NAV using Buildbot John Magne ---------- * power supply trap * arnold improvements Kai-Arne -------- * gather netbios names Scheduled for 3.8 ================= * dns name, syslocation, etc (Morten) * networkdiscovery to python (Morten) * wireless controller support (Fredrik?) Should come along in a later 3.6 beta: ====================================== * Trigger alarms on certain mac addresses * ipdevpoll * modulemon * algorithm: remove module, insert new * problem: interfaces are never deleted... (gone since is set) * do not poll when a box is down * delete swportallowed vlan when a port is changed trunk->no_trunk * add mem info from cisco devices * table & plugin for environmental sensor * implement static routes? or leave out... * if a box has a change in serial - remove info and collect new data * support the ability to except certain prefixes (in ipdevpoll.conf) -- - Vidar

1 1

Database version control
by Morten Brekkevold 26 Mai '10

26 Mai '10

In reference to my database related mess during the first beta release of 3.6.0, I want to propose a few changes to how we manage schema changes in NAV. We do version control our schema, which is good, but the process has some weaknesses. The fact that we maintain a baseline and change scripts separately at the same time has repeatedly caused inconsistencies. Usually, I catch these problems with a simple schema diffing tool, but since we still have no automated tests going on, this is a tedious manual task and prone to being forgotten. I found a series of articles about database version control via the Coding Horror blog. These describe many of the ideas I was having as I was fixing the b1 release: http://www.codinghorror.com/blog/2008/02/get-your-database-under-version-co… Basic summary of the changes I'm suggesting: * We keep the sql scripts, doc/sql/*.sql, as they are - these are our baseline. * Instead of everyone maintaining both doc/sql/upgrades/trunk.sql and the baseline scripts: Whenever a developer needs to change the schema, he/she should check in a new sql scrip which contains only the ALTER/UPDATE statements needed to migrate to the new schema. * Schema initialization and upgrade are merged into one automatic process that applies the baseline, and then consecutively the change scripts committed since the baseline. * To keep track of which schema changes have been installed, a new schema_change_log table in the database is updated by the automagic schema install/upgrade process. * Every once in a while, typically at a new feature release, the schema change files can be summarized into a new baseline. This process would also make it easier to install an arbitrary revision from a Mercurial repo and ensure that one has the correct schema for that revision. I guess we will be busy prepping the final 3.6.0 for some time yet, but we should be looking into this post 3.6 (not to mention all the automatic testing I'm dreaming of putting togheter post 3.6). Feedback/comments are appreciated, though I know many of you are busy with non-NAV-related stuff this time of year. -- Morten Brekkevold UNINETT

1 0

Notes on the beta release and branch changes
by Morten Brekkevold 21 Mai '10

21 Mai '10

Hi all devs, Just a quick note on the ongoing changes in relation to the 3.6 series. The staging/3.6 branch has now been merged to default. A new series/3.6.x branch has been created. The series/3.6.x is not considered stable yet, so all current devs have push access to this repository. * If you need to commit changes to the 3.6 development series, please clone the series/3.6.x repo to do so. Remember that this is a named branch. Also, for those who haven't noticed yet, the entire build system has switched from pure autoconf to the automake+autoconf combination. All Makefile.inS have been rewritten to Makefile.amS, reducing heaps of redundancies and making the Makefiles more readable. Please refer to the GNU autotools manuals for further info (or start a discussion here or offline if you have comments/questions). -- Morten Brekkevold UNINETT

2 1

Meeting minutes from 2010-05-03
by Kristian Klette 18 Mai '10

18 Mai '10

Here are the meeting minutes from the last NAV developer meeting. Sorry about the delay. NAV Meeting May 3 2010 -- NAV 3.6 Release ========================================= Present: - Vidar - John Magne - Morten - Kristian - Thomas - Magnus Discussions regarding NAV 3.6 ============================= Deadline 2010-05-14 moved to 2010-05-21. Beta release: 2010-05-21 LP has: * 36 blueprints * 13 bugs targeted for 3.6 Blueprints ---------- - ipdevpoll - Beta is just around the corner. - Some plugins aren't ready yet. These will be included during the beta period. - GetBoksMacs - Good progress, needs more testing before released into the wild. - Replace Ip info by extending IP Device Info - Feedback from Jørgen needed. - Device History - Done. The toolbox description still has the "beta"-mark. Should be removed. - Magnus wants to clean up the code, but not critical for 3.6 - API for adding custom event/alert type hierarchy - Done. - Add names to identify modules - Done. - Modules uses the new "name" attribute for identifying the module. - The old numeric ids are kept and used for sorting. - CSV export - Done - ThresholdMon refactor - Done - Refactor nav.auth to be django aware - Almost done. - LDAP authentication needs testing. - Should switch session cookie identifier for smoother upgrade path for users. - Remove SQL parsing from report generator - Done - Remove psycopg1 dependency in favor of pyscopg2 - Done - Might be some issues regarding unicode in legacy code. - Rewrite pping and servicemon to use nav.deamon - Done - Rewrite makecricketconf to python / Rename RRD-files when changing target name - Ready for beta. - Upgrade paths regarding rrd files must be tested. - Check and refactor statTools to support consolidated interface table - Obsolete - Collect WeatherGoose statistics with cricket - Started - Soon ready, but there are some issues with detecting weathergoose devices thats needs to be sorted out. - Rewrite status page to django - Done. - Needs code-review and testing Bugs ---- - Prefixes not removed - Should be fixed in ipdevpoll. - Missing alertmsg.conf for cisco AP trap - Fixed in JM's staging branch. - Fix should be cherry-picked into 3.5.x - Alert when mac-address connects to the network (wishlist) - Moved to NAV 3.7 - Default location and organisation in install (wishlist) - Installations that only has one organization (or simply doesn't need org.-tracking) wants to have a default dummy organization for easier seeding of the database. -- Mvh Kristian Klette «Programs for sale: Fast, Reliable, Cheap: choose two.»

1 0

NAV status meeting minutes 2009-11-23
by Kristian Klette 23 Nov '09

23 Nov '09

NAV status meeting minutes 2009-11-23 ===================================== Present: Magnus, Vidar, Kristian Absent: Morten, Jørgen, Thomas 3.6.0 status * 6 blueprints not yet done or not even started. Magnus ------- - Completed Webfront refactoring. - Fixed a lot of minor issues caused by the webfront refactoring. Mostly template rendering issues. - Fixed DeviceHistory sorting. - Added sane defaults and a new design to StatusPreferences. Klette ------ - Netmap refactoring completed. - NetworkExplorer refactoring completed. - Fixed some minor bugs in NetworkDiscovery. - Started on ipdevinfo-blueprint (blueprint stolen from Jørgen). etc --- - Morten will be absent the next two weeks or so due to a new child. Congratulations from the entire NAV-team! -- Mvh Kristian Klette «Programs for sale: Fast, Reliable, Cheap: choose two.»

1 0

Patch for StatusSections.py (problem with postgres-8.3)
by Asbjørn Prøis 18 Nov '09

18 Nov '09

Attached is a small patch for making StatusSections.py work on NAV systems running with postgres-8.3. -Asbjørn-

1 0

Merged magnusme-refactor-auth to default branch
by Morten Brekkevold 10 Nov '09

10 Nov '09

FYI: I just merged Magnus' refactor-auth branch onto the default branch. This entails that session['user'] is no longer a forgetSQL object, but a dictionary. The dictionary will contain the keys 'id', 'login' and 'name'. The branch (hopefully) refactors all NAV code that expected a forgetSQL account object. -- mvh Morten Brekkevold UNINETT

1 0

Re-engineering the OID database
by Morten Brekkevold 05 Nov '09

05 Nov '09

While designing ipdevpoll, we decided to throw out the over-engineered OID database from NAV. The tables haven't been removed yet, but none of the new code makes use of them. Now, it looks like we need to keep some aspects of the OID database. While configuring Cricket, makecricketconfig needs to know ahead of time which proprietary OIDs from the Cricket config files are supported by an IP device. In other words, makecricketconfig still needs to have an SNMP OID profile for a box. John-Magne has written a script that will pull all configured OID labels from the Cricket configuration tree, and insert them into NAV's snmpoid table. I've written an oidprofiler plugin to ipdevpoll, which does most of what getDeviceData's OID tester did. But, some superfluous fields can be removed from the snmpoid and netboxsnmpoid tables to complete the transition to NAV 3.6: * The netboxsnmpoid.frequency and snmpoid.defaultfreq fields are right out, since these tables will no longer be used for scheduling anything. * snmpoid.oidsource is irrelevant in the new context. * snmpoid.descr, oidname and mib can be safely removed. If anyone needs them, they can be inferred by a mib compiler. * snmpoid.decodehex is just ridiculous. * snmpoid.match_regex is unnecessary, because there is no such information in the Cricket config files. And the more interesting part: * snmpoid.uptodate - We can't really assess whether an snmpoid is really up-to-date or not. When gDD sees an snmpoid with uptodate=false, it will query that snmpoid for all netboxes and set uptodate=true. But "all netboxes" only means all netboxes that are online at the moment. There is no guarantee that all netboxes were actually queried. My suggestion is that we rather set netbox.uptodate=false for all netboxes when snmpoids are inserted or updated. Re-profiling a netbox is not a huge operation (especially since the current cricket config contains about 12 OIDs that need to be profiles), and new snmpoids will not be inserted very often. This can be implemented via a simple trigger or rule on the snmpoid table. This is FYI, and a heads-up for a possible discussion on Monday's status meeting. -- Morten Brekkevold UNINETT

1 0

← Newer
1
2
3
4
5
6
7
8
9
10
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Nav-dev