Nav-dev April 2009

nav-dev@lister.sikt.no

1 participants
2 discussions

Deprecating psycopg1
by Morten Brekkevold 22 Apr '09

22 Apr '09

Hi all! As mentioned on the IRC channel before easter week, I've been taking a shot at the deprecate-psycopg1 blueprint (https://blueprints.launchpad.net/nav/+spec/deprecate-psycopg1) Preliminary results can be found on this branch: http://metanav.uninett.no/hg/features/deprecate-psycopg1/ I would like to merge this code with the default branch as soon as possible. I would like it, though, if people who feel they have some sort of responsibility for code they've written would review the diffs. A lot of code was using the dictfetchall, dictfetchone and dictfetchmany call on cursors. These are non-standard, and are no longer supported in psycopg2. To avoid making too many code changes related to this, I've opted to use a psycopg2 extension which allows me to use customized cursor classes. The psycopg2 extensions already comes with a custom cursor class, which provides result rows that can be accessed both as tuples and dictionaries. To use the DictCursor class, I've had to manually specify the class as an argument to every call to connection.cursor(), where the code later expected to use the dictfetch methods on the cursor. All dictfetch calls have been reduced to regular fetch calls. The custom cursor change affects the following subsystems: * Arnold * deviceManagement * ipinfo * l2trace * lib-python (rrdPresenter) * maintenance * messages * Netmap * snmptrapd * statTools See this changeset to see how I may have mauled your code: http://metanav.uninett.no/hg/features/deprecate-psycopg1/rev/4cbc70349d55 On another note: Does all this code _really_ need to use dictionary result rows? I don't know the internals of psycopg2's DictCursor, but I seems to me that it could be a real memory-waste to pull thousands of table rows as dictionary instances instead of simple tuples. In many cases, I think looping over the results like this would suffice: cursor.execute("SELECT column1, column2, column3 FROM foo") for (column1, column2, column3) in cursor.fetchall(): pass # or actually do something -- mvh Morten Brekkevold UNINETT

1 3

NAV status meeting minutes 2009-03-23
by Morten Brekkevold 02 Apr '09

02 Apr '09

My apologies, I keep forgetting to post the meeting minutes to the list. On request, here are the minutes for last week's status meeting. Since next week is easter week, the next meeting will likely be April 21st, to stay on odd-numbered weeks. NAV status meeting minutes 2009-03-23 ===================================== morten ------ * released NAV 3.5.2 with two security fixes, coordinated with Werner for simultaneous release of the Debian package. * fixed a couple of ipdevinfo bugs (rrd-availability & downtime for stateless alerts) * Fixed Netmap UTF-8 bug for Windows users. Kristian comments that the old convoluted code was an attempted workaround for some older java plugins. My take is that we should recommend everyone always use the latest available version. * using virtualbox on my workstation for deployment tests, old sdrift machine decommissioned. * new virtual machine, elixis, for ipdevpoll development. * "apache idle in transactions"-problem on client installations fixed. Was related to Apache site config overriding NAV's .htaccess file. * consolidated interface table looks complete. Schema upgrade statements written and tested. Should have a short meeting about this when Vidar gets back fro Lisbon. jørgen ------ * working on smsd backoff mechanism magnus ------ * wrote a Django base template to have SQL query debug info on all pages in Django subsystems. * Hacking on error message logging in IP Device history. Found a quickselect defect: Cannot explicitly set a default selection, e.g. when form needs to be redisplayed because of errors in form input. thomas ------ * alertengine bugfixing. discussion ---------- Questions about priorities: Magnus: Priority list: 1=finish ipdevhist, 2=ipdevpoll tasks, if any are assigned, 3=refactor-auth blueprint Jørgen: Priority list: 1=smsd backoff, 2=ipdevinfo-related bugs and blueprints, 3=report-sql-parse-refactor -- mvh Morten Brekkevold UNINETT

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Nav-dev April 2009