Hello all,
Great work on NAV, appreciate all the hard work.
I've looked through the documentation and searched the archive and I haven't seen this issue. I can't seem to pin down the cause. NAV version is 4.1.2 on the Debian virtual appliance you offer. I've added a new group of Cisco 3850s to a network and now changes to port information aren't saved. Description information is not saved at all. VLAN information is changed, but only ephemerally, not in the actual port configuration. This means that the VLAN information isn't preserved across reloads. The only difference that I can find are the IOS version and VTP. The previous 3850s are running 3.3.x and the new 3850s are running 3.6.1. The new 3850s are using VTP v3 whereas the current 3850s are not.
There are no error messages generated during the port configuration that I can find.
Has anyone seen this before?
Thanks.
On Tue, 25 Nov 2014 17:40:58 +0100 (CET) kentobias@wcn.co wrote:
Great work on NAV, appreciate all the hard work.
Hi, and thanks!
I've added a new group of Cisco 3850s to a network and now changes to port information aren't saved. Description information is not saved at all.
When you are talking about changes to port information and description, are you talking about NAV's collection of these, or of configuring these through NAV's Port Admin tool? I'm guessing the latter...
VLAN information is changed, but only ephemerally, not in the actual port configuration. This means that the VLAN information isn't preserved across reloads.
Some operations are vendor proprietary in Port Admin - we use Cisco proprietary MIBs to achieve VLAN changes to Cisco ports, and also to instruct the device to perform a "write mem" after changes have been made.
The only difference that I can find are the IOS version and VTP. The previous 3850s are running 3.3.x and the new 3850s are running 3.6.1. The new 3850s are using VTP v3 whereas the current 3850s are not.
Do your new 3850s and your old 3850s share the same device type in NAV? (Take a look at them in IP Device Info, and click their "Type" descriptions).
Sometimes, newer software or hardware revisions will have different SNMP sysObjectID's. If the new devices reported sysObjectID's unknown to NAV, ipdevpoll will have autocreated new device types for these, attaching them to the "unknown" vendor id.
There is a pitfall here: Port Admin will look at a device's type's vendor id to determine which operations to use to perform its work. If the device type's vendor id is not "cisco", it will not use Cisco proprietary MIBs to perform its changes.
If this is your problem, you can clean up the type definitions using SeedDB.
There are no error messages generated during the port configuration that I can find.
I cannot speculate as to why no error messages were generated. I'm not that familiar with the inner workings of Port Admin - we'd have to do some testing.
Sorry I wasn't clear in the first post. The changes I refer to are those made with the NAV Port Admin Tool. The objective being to have the "description" and "switchport access vlan ???" values change on individual ports.
The OID IS different for the new 3850s: 1.3.6.1.4.1.9.1.1745. The older version is: 1.3.6.1.4.1.9.1.1641. I created a new device type in the Seed DB with the existing vendor "cisco", which is the same as the old device type. Is there more I have to do to have NAV recognize the new device type as Cisco?
On Wed, Nov 26, 2014 at 11:57 PM, Morten Brekkevold < morten.brekkevold@uninett.no> wrote:
On Tue, 25 Nov 2014 17:40:58 +0100 (CET) kentobias@wcn.co wrote:
Great work on NAV, appreciate all the hard work.
Hi, and thanks!
I've added a new group of Cisco 3850s to a network and now changes to port information aren't saved. Description information is not saved at all.
When you are talking about changes to port information and description, are you talking about NAV's collection of these, or of configuring these through NAV's Port Admin tool? I'm guessing the latter...
VLAN information is changed, but only ephemerally, not in the actual port configuration. This means that the VLAN information isn't preserved across reloads.
Some operations are vendor proprietary in Port Admin - we use Cisco proprietary MIBs to achieve VLAN changes to Cisco ports, and also to instruct the device to perform a "write mem" after changes have been made.
The only difference that I can find are the IOS version and VTP. The previous 3850s are running 3.3.x and the new 3850s are running 3.6.1. The new 3850s are using VTP v3 whereas the current 3850s are not.
Do your new 3850s and your old 3850s share the same device type in NAV? (Take a look at them in IP Device Info, and click their "Type" descriptions).
Sometimes, newer software or hardware revisions will have different SNMP sysObjectID's. If the new devices reported sysObjectID's unknown to NAV, ipdevpoll will have autocreated new device types for these, attaching them to the "unknown" vendor id.
There is a pitfall here: Port Admin will look at a device's type's vendor id to determine which operations to use to perform its work. If the device type's vendor id is not "cisco", it will not use Cisco proprietary MIBs to perform its changes.
If this is your problem, you can clean up the type definitions using SeedDB.
There are no error messages generated during the port configuration that I can find.
I cannot speculate as to why no error messages were generated. I'm not that familiar with the inner workings of Port Admin - we'd have to do some testing.
-- Morten Brekkevold UNINETT
On Thu, 27 Nov 2014 08:47:02 -0800 Ken Tobias kentobias@wcn.co wrote:
The OID IS different for the new 3850s: 1.3.6.1.4.1.9.1.1745. The older version is: 1.3.6.1.4.1.9.1.1641. I created a new device type in the Seed DB with the existing vendor "cisco", which is the same as the old device type. Is there more I have to do to have NAV recognize the new device type as Cisco?
If you manually created a device type with this sysObjectID after my last post, you will have created a duplicate (not sure how you were able to, though - maybe because the sysObjectIDs are normally stored with a preceding dot?).
ipdevpoll will have automatically created a new type for .1.3.6.1.4.1.9.1.1745 as soon as it was seen. This type will have that very same value (.1.3.6.1.4.1.9.1.1745) as its typename, and the sysDecr value of the device it was first discovered on will have been used as its description. "unknown" will be the vendor.
You need to find this auto-created type using the SeedDB tool. Edit its name and description into something more useful, and change its vendor from "unknown" to "cisco". That should be all you need to do for Port Admin to use the proper Cisco MIBs for configuring VLANs on your new 3850s.
I had originally modified the auto-created "unknown" type to be the new 3850 type. There is no "unknown" type in the DB.
I have upgraded NAV to 4.2.1 and I now get one the following errors (randomly) on the port configuration page when attempting to set the VLAN / description on a port on the new 3850:
Error setting vlan: SNMP Error: wrongType (The set datatype does not match the data type the agent expects)
Error setting vlan: Timed out waiting for SNMP response: Timeout Error setting ifalias: Timed out waiting for SNMP response: Timeout
Thanks!
On Fri, Nov 28, 2014 at 1:26 AM, Morten Brekkevold < morten.brekkevold@uninett.no> wrote:
On Thu, 27 Nov 2014 08:47:02 -0800 Ken Tobias kentobias@wcn.co wrote:
The OID IS different for the new 3850s: 1.3.6.1.4.1.9.1.1745. The older version is: 1.3.6.1.4.1.9.1.1641. I created a new device type in the Seed DB with the existing vendor "cisco", which is the same as the old device type. Is there more I have to do to have NAV recognize the new device
type
as Cisco?
If you manually created a device type with this sysObjectID after my last post, you will have created a duplicate (not sure how you were able to, though - maybe because the sysObjectIDs are normally stored with a preceding dot?).
ipdevpoll will have automatically created a new type for .1.3.6.1.4.1.9.1.1745 as soon as it was seen. This type will have that very same value (.1.3.6.1.4.1.9.1.1745) as its typename, and the sysDecr value of the device it was first discovered on will have been used as its description. "unknown" will be the vendor.
You need to find this auto-created type using the SeedDB tool. Edit its name and description into something more useful, and change its vendor from "unknown" to "cisco". That should be all you need to do for Port Admin to use the proper Cisco MIBs for configuring VLANs on your new 3850s.
-- Morten Brekkevold UNINETT
Hi Ken!
On 01. des. 2014 20:01, Ken Tobias wrote:
I had originally modified the auto-created "unknown" type to be the new 3850 type. There is no "unknown" type in the DB.
Just to be absolutely sure that we are understanding each other we would like you to attach a screenshot of the type you edited. Just edit the type again and take a screenshot of the form.
I have upgraded NAV to 4.2.1 and I now get one the following errors (randomly) on the port configuration page when attempting to set the VLAN / description on a port on the new 3850:
Random errors are the worst! However, some of the actions in PortAdmin are logged to the Apache error log. If you could post what the logs say when the errors occur, that would perhaps help us.
Here is the screen clip.
[image: Inline image 1]
This is the text from /var/log/apache2/error.log
[Wed Dec 03 12:49:04 2014] [error] [Wed Dec 03 12:49:04 2014] [INFO] [pid=2347 nav.web.portadmin] ktobias: wladc-acc-sw-01:Gi2/0/2 - vlan set to 1251 [Wed Dec 03 12:49:08 2014] [error] [Wed Dec 03 12:49:08 2014] [ERROR] [pid=2347 nav.web.portadmin] Error doing write mem on wladc-acc-sw-01: Timed out waiting for SNMP response: Timeout
No other log has error data. other_vhosts_access.log has a record of my "POST".
Let me know if there is other data I can pull.
On Wed, Dec 3, 2014 at 5:15 AM, John Magne Bredal john.m.bredal@uninett.no wrote:
Hi Ken!
On 01. des. 2014 20:01, Ken Tobias wrote:
I had originally modified the auto-created "unknown" type to be the new 3850 type. There is no "unknown" type in the DB.
Just to be absolutely sure that we are understanding each other we would like you to attach a screenshot of the type you edited. Just edit the type again and take a screenshot of the form.
I have upgraded NAV to 4.2.1 and I now get one the following errors (randomly) on the port configuration page when attempting to set the VLAN / description on a port on the new 3850:
Random errors are the worst! However, some of the actions in PortAdmin are logged to the Apache error log. If you could post what the logs say when the errors occur, that would perhaps help us.
-- John Magne Bredal john.m.bredal@uninett.no +4791897366
Abels gt. 5- Teknobyen NO-7465 Trondheim
On 03. des. 2014 21:53, Ken Tobias wrote:
Here is the screen clip.
Inline image 1
Thank you. That looks quite correct.
This is the text from /var/log/apache2/error.log
[Wed Dec 03 12:49:04 2014] [error] [Wed Dec 03 12:49:04 2014] [INFO] [pid=2347 nav.web.portadmin] ktobias: wladc-acc-sw-01:Gi2/0/2 - vlan set to 1251 [Wed Dec 03 12:49:08 2014] [error] [Wed Dec 03 12:49:08 2014] [ERROR] [pid=2347 nav.web.portadmin] Error doing write mem on wladc-acc-sw-01: Timed out waiting for SNMP response: Timeout
No other log has error data. other_vhosts_access.log has a record of my "POST".
Let me know if there is other data I can pull.
This was helpful, thanks.
I think we're down to testing SNMP commands from cli now unfortunately. The object we use to do a write memory on Cisco devices is OLD-CISCO-SYS-MIB::writeMem (1.3.6.1.4.1.9.2.1.54) .
If you could try something like this to verify that this fails (this tries to do a 'write mem' on the device):
snmpset -c <write-community> -v2c wladc-acc-sw-01 1.3.6.1.4.1.9.2.1.54.0 i 1
The interesting part is if adjusting the timeout with -t <seconds> helps in any way. Try for instance to set the timeout to 5 or 10 seconds.
On Wed, Dec 3, 2014 at 5:15 AM, John Magne Bredal <john.m.bredal@uninett.no mailto:john.m.bredal@uninett.no> wrote:
Hi Ken! On 01. des. 2014 20:01, Ken Tobias wrote: > I had originally modified the auto-created "unknown" type to be the new > 3850 type. There is no "unknown" type in the DB. Just to be absolutely sure that we are understanding each other we would like you to attach a screenshot of the type you edited. Just edit the type again and take a screenshot of the form. > I have upgraded NAV to 4.2.1 and I now get one the following errors > (randomly) on the port configuration page when attempting to set the > VLAN / description on a port on the new 3850: Random errors are the worst! However, some of the actions in PortAdmin are logged to the Apache error log. If you could post what the logs say when the errors occur, that would perhaps help us. -- John Magne Bredal john.m.bredal@uninett.no <mailto:john.m.bredal@uninett.no> +4791897366 <tel:%2B4791897366> Abels gt. 5- Teknobyen NO-7465 Trondheim
I will give this a try today.
As a footnote. On a 3850 stack, A "write mem" takes quite a while as it replicates the command across each device since each one can become that active stack controller. From the IOS CLI it can take 5-10 seconds normally.
On Thu, Dec 4, 2014 at 2:35 AM, John Magne Bredal john.m.bredal@uninett.no wrote:
On 03. des. 2014 21:53, Ken Tobias wrote:
Here is the screen clip.
Inline image 1
Thank you. That looks quite correct.
This is the text from /var/log/apache2/error.log
[Wed Dec 03 12:49:04 2014] [error] [Wed Dec 03 12:49:04 2014] [INFO] [pid=2347 nav.web.portadmin] ktobias: wladc-acc-sw-01:Gi2/0/2 - vlan set to 1251 [Wed Dec 03 12:49:08 2014] [error] [Wed Dec 03 12:49:08 2014] [ERROR] [pid=2347 nav.web.portadmin] Error doing write mem on wladc-acc-sw-01: Timed out waiting for SNMP response: Timeout
No other log has error data. other_vhosts_access.log has a record of my "POST".
Let me know if there is other data I can pull.
This was helpful, thanks.
I think we're down to testing SNMP commands from cli now unfortunately. The object we use to do a write memory on Cisco devices is OLD-CISCO-SYS-MIB::writeMem (1.3.6.1.4.1.9.2.1.54) .
If you could try something like this to verify that this fails (this tries to do a 'write mem' on the device):
snmpset -c <write-community> -v2c wladc-acc-sw-01 1.3.6.1.4.1.9.2.1.54.0 i 1
The interesting part is if adjusting the timeout with -t <seconds> helps in any way. Try for instance to set the timeout to 5 or 10 seconds.
On Wed, Dec 3, 2014 at 5:15 AM, John Magne Bredal <john.m.bredal@uninett.no mailto:john.m.bredal@uninett.no> wrote:
Hi Ken! On 01. des. 2014 20:01, Ken Tobias wrote: > I had originally modified the auto-created "unknown" type to bethe new
> 3850 type. There is no "unknown" type in the DB. Just to be absolutely sure that we are understanding each other wewould
like you to attach a screenshot of the type you edited. Just edit the type again and take a screenshot of the form. > I have upgraded NAV to 4.2.1 and I now get one the following errors > (randomly) on the port configuration page when attempting to setthe
> VLAN / description on a port on the new 3850: Random errors are the worst! However, some of the actions inPortAdmin
are logged to the Apache error log. If you could post what the logssay
when the errors occur, that would perhaps help us. -- John Magne Bredal john.m.bredal@uninett.no <mailto:john.m.bredal@uninett.no> +4791897366 <tel:%2B4791897366> Abels gt. 5- Teknobyen NO-7465 Trondheim-- John Magne Bredal john.m.bredal@uninett.no +4791897366
Abels gt. 5- Teknobyen NO-7465 Trondheim
The output of the "snmpset" command with "-t <#>" values ranging from 1-30 seconds is:
iso.3.6.1.4.1.9.2.1.54.0 = INTEGER: 1
It returns without any noticeable delay no matter what the timeout value is.
I don't know if this is relevant but when the port config changes are "saved" from NAV the result is often that the values are set in some ephemeral way, just not in the config file. For example, doing a "sh vlan" will show the ports in question being in the desired VLAN after the "save" despite the running config not reflecting the change. The timeout responses occur when trying to set multiple values (e.g. description and VLAN, or multiple port VLAN changes) or when executing multiple changes in a short time frame. Are there multiple OIDs for setting values permanently vs. temporarily?
On Thu, Dec 4, 2014 at 9:12 AM, Ken Tobias kentobias@wcn.co wrote:
I will give this a try today.
As a footnote. On a 3850 stack, A "write mem" takes quite a while as it replicates the command across each device since each one can become that active stack controller. From the IOS CLI it can take 5-10 seconds normally.
On Thu, Dec 4, 2014 at 2:35 AM, John Magne Bredal < john.m.bredal@uninett.no> wrote:
On 03. des. 2014 21:53, Ken Tobias wrote:
Here is the screen clip.
Inline image 1
Thank you. That looks quite correct.
This is the text from /var/log/apache2/error.log
[Wed Dec 03 12:49:04 2014] [error] [Wed Dec 03 12:49:04 2014] [INFO] [pid=2347 nav.web.portadmin] ktobias: wladc-acc-sw-01:Gi2/0/2 - vlan set to 1251 [Wed Dec 03 12:49:08 2014] [error] [Wed Dec 03 12:49:08 2014] [ERROR] [pid=2347 nav.web.portadmin] Error doing write mem on wladc-acc-sw-01: Timed out waiting for SNMP response: Timeout
No other log has error data. other_vhosts_access.log has a record of my "POST".
Let me know if there is other data I can pull.
This was helpful, thanks.
I think we're down to testing SNMP commands from cli now unfortunately. The object we use to do a write memory on Cisco devices is OLD-CISCO-SYS-MIB::writeMem (1.3.6.1.4.1.9.2.1.54) .
If you could try something like this to verify that this fails (this tries to do a 'write mem' on the device):
snmpset -c <write-community> -v2c wladc-acc-sw-01 1.3.6.1.4.1.9.2.1.54.0 i 1
The interesting part is if adjusting the timeout with -t <seconds> helps in any way. Try for instance to set the timeout to 5 or 10 seconds.
On Wed, Dec 3, 2014 at 5:15 AM, John Magne Bredal <john.m.bredal@uninett.no mailto:john.m.bredal@uninett.no> wrote:
Hi Ken! On 01. des. 2014 20:01, Ken Tobias wrote: > I had originally modified the auto-created "unknown" type to bethe new
> 3850 type. There is no "unknown" type in the DB. Just to be absolutely sure that we are understanding each other wewould
like you to attach a screenshot of the type you edited. Just editthe
type again and take a screenshot of the form. > I have upgraded NAV to 4.2.1 and I now get one the followingerrors
> (randomly) on the port configuration page when attempting to setthe
> VLAN / description on a port on the new 3850: Random errors are the worst! However, some of the actions inPortAdmin
are logged to the Apache error log. If you could post what the logssay
when the errors occur, that would perhaps help us. -- John Magne Bredal john.m.bredal@uninett.no <mailto:john.m.bredal@uninett.no> +4791897366 <tel:%2B4791897366> Abels gt. 5- Teknobyen NO-7465 Trondheim-- John Magne Bredal john.m.bredal@uninett.no +4791897366
Abels gt. 5- Teknobyen NO-7465 Trondheim
On 04. des. 2014 19:10, Ken Tobias wrote:
The output of the "snmpset" command with "-t <#>" values ranging from 1-30 seconds is:
iso.3.6.1.4.1.9.2.1.54.0 = INTEGER: 1
It returns without any noticeable delay no matter what the timeout value is.
Ok, thanks for testing that.
I don't know if this is relevant but when the port config changes are "saved" from NAV the result is often that the values are set in some ephemeral way, just not in the config file. For example, doing a "sh vlan" will show the ports in question being in the desired VLAN after the "save" despite the running config not reflecting the change.
As far as I know the running config is the non-persistent config, and the startup config is the persistent config. So what you say is that the result of the command 'show run' and 'show vlan' does not display the same information, and for that I have no real explanation. I will ask around and see if anyone else have experienced this.
The timeout responses occur when trying to set multiple values (e.g. description and VLAN, or multiple port VLAN changes) or when executing multiple changes in a short time frame. Are there multiple OIDs for setting values permanently vs. temporarily?
I see. Looking through the code it seems that a write mem is done for each changed interface. This is clearly a bad way of doing it, and also explains some of your issues.
I have created an issue and a patch for this that can be found at https://bugs.launchpad.net/nav/+bug/1399558
It is true that the startup config is the non-volatile version of the config that will be loaded after a reboot or power fail. However all changes are first stored in volatile storage in the running config. A write mem copies the running config to the startup config. If the change isn't in the running config before a write mem, it's not going to be in the startup config after a write mem. This is why I brought it up. If the setting of the values was successful but the write mem was not, it should show up in the running config but not the startup config. This implies that the values were never effectively set in the running config. I've read about dynamic VLANs that are set at run time based on policy. They don't show up in the running config or startup config to the best of my knowledge. That is the only scenario I can think of that would result in a port being added to the vlan tables but not the config file. If we are using the "old Cisco MIB", maybe Cisco changed the default function of that OID to map to dynamic VLAN changes instead of static. Are there more current OIDs that let you choose which type of setting is preferred? Just a guess as this is not my specialty. I did a little looking around in the MIB and there are indeed other/newer OIDs for this, but I couldn't get them to work properly with snmpget and snmpset.
On Fri, Dec 5, 2014 at 12:42 AM, John Magne Bredal <john.m.bredal@uninett.no
wrote:
On 04. des. 2014 19:10, Ken Tobias wrote:
The output of the "snmpset" command with "-t <#>" values ranging from 1-30 seconds is:
iso.3.6.1.4.1.9.2.1.54.0 = INTEGER: 1
It returns without any noticeable delay no matter what the timeout value is.
Ok, thanks for testing that.
I don't know if this is relevant but when the port config changes are "saved" from NAV the result is often that the values are set in some ephemeral way, just not in the config file. For example, doing a "sh vlan" will show the ports in question being in the desired VLAN after the "save" despite the running config not reflecting the change.
As far as I know the running config is the non-persistent config, and the startup config is the persistent config. So what you say is that the result of the command 'show run' and 'show vlan' does not display the same information, and for that I have no real explanation. I will ask around and see if anyone else have experienced this.
The timeout responses occur when trying to set multiple values (e.g. description and VLAN, or multiple port VLAN changes) or when executing multiple changes in a short time frame. Are there multiple OIDs for setting values permanently vs. temporarily?
I see. Looking through the code it seems that a write mem is done for each changed interface. This is clearly a bad way of doing it, and also explains some of your issues.
I have created an issue and a patch for this that can be found at https://bugs.launchpad.net/nav/+bug/1399558
-- John Magne Bredal john.m.bredal@uninett.no +4791897366
Abels gt. 5- Teknobyen NO-7465 Trondheim
On 05. des. 2014 17:24, Ken Tobias wrote:
It is true that the startup config is the non-volatile version of the config that will be loaded after a reboot or power fail. However all changes are first stored in volatile storage in the running config. A write mem copies the running config to the startup config. If the change isn't in the running config before a write mem, it's not going to be in the startup config after a write mem. This is why I brought it up.
Agree.
If the setting of the values was successful but the write mem was not, it should show up in the running config but not the startup config. This implies that the values were never effectively set in the running config. I've read about dynamic VLANs that are set at run time based on policy. They don't show up in the running config or startup config to the best of my knowledge. That is the only scenario I can think of that would result in a port being added to the vlan tables but not the config file.
That is interesting.
If we are using the "old Cisco MIB", maybe Cisco changed the default function of that OID to map to dynamic VLAN changes instead of static. Are there more current OIDs that let you choose which type of setting is preferred? Just a guess as this is not my specialty. I did a little looking around in the MIB and there are indeed other/newer OIDs for this, but I couldn't get them to work properly with snmpget and snmpset.
We are very interested in the OIDs you found. One problem we have at the moment is that we have no way of testing this in an exact matter as we have no 3850.
I have asked around some and am expecting some feedback shortly. I will let you know what we discover.
Thanks. Appreciated.
On Mon, Dec 8, 2014 at 3:05 AM, John Magne Bredal john.m.bredal@uninett.no wrote:
On 05. des. 2014 17:24, Ken Tobias wrote:
It is true that the startup config is the non-volatile version of the config that will be loaded after a reboot or power fail. However all changes are first stored in volatile storage in the running config. A write mem copies the running config to the startup config. If the change isn't in the running config before a write mem, it's not going to be in the startup config after a write mem. This is why I brought it up.
Agree.
If the setting of the values was successful but the write mem was not, it should show up in the running config but not the startup config. This implies that the values were never effectively set in the running config. I've read about dynamic VLANs that are set at run time based on policy. They don't show up in the running config or startup config to the best of my knowledge. That is the only scenario I can think of that would result in a port being added to the vlan tables but not the config file.
That is interesting.
If we are using the "old Cisco MIB", maybe Cisco changed the default function of that OID to map to dynamic VLAN changes instead of static. Are there more current OIDs that let you choose which type of setting is preferred? Just a guess as this is not my specialty. I did a little looking around in the MIB and there are indeed other/newer OIDs for this, but I couldn't get them to work properly with snmpget and snmpset.
We are very interested in the OIDs you found. One problem we have at the moment is that we have no way of testing this in an exact matter as we have no 3850.
I have asked around some and am expecting some feedback shortly. I will let you know what we discover.
-- John Magne Bredal john.m.bredal@uninett.no +4791897366
Abels gt. 5- Teknobyen NO-7465 Trondheim
The OIDs I found that seemed related were:
- 1.3.6.1.4.1.9.9.96 - 1.3.6.1.4.1.9.5.1.29.3 - 1.3.6.1.4.1.9.5.1.29.4
On Mon, Dec 8, 2014 at 12:25 PM, Ken Tobias kentobias@wcn.co wrote:
Thanks. Appreciated.
On Mon, Dec 8, 2014 at 3:05 AM, John Magne Bredal < john.m.bredal@uninett.no> wrote:
On 05. des. 2014 17:24, Ken Tobias wrote:
It is true that the startup config is the non-volatile version of the config that will be loaded after a reboot or power fail. However all changes are first stored in volatile storage in the running config. A write mem copies the running config to the startup config. If the change isn't in the running config before a write mem, it's not going to be in the startup config after a write mem. This is why I brought it up.
Agree.
If the setting of the values was successful but the write mem was not, it should show up in the running config but not the startup config. This implies that the values were never effectively set in the running config. I've read about dynamic VLANs that are set at run time based on policy. They don't show up in the running config or startup config to the best of my knowledge. That is the only scenario I can think of that would result in a port being added to the vlan tables but not the config file.
That is interesting.
If we are using the "old Cisco MIB", maybe Cisco changed the default function of that OID to map to dynamic VLAN changes instead of static. Are there more current OIDs that let you choose which type of setting is preferred? Just a guess as this is not my specialty. I did a little looking around in the MIB and there are indeed other/newer OIDs for this, but I couldn't get them to work properly with snmpget and snmpset.
We are very interested in the OIDs you found. One problem we have at the moment is that we have no way of testing this in an exact matter as we have no 3850.
I have asked around some and am expecting some feedback shortly. I will let you know what we discover.
-- John Magne Bredal john.m.bredal@uninett.no +4791897366
Abels gt. 5- Teknobyen NO-7465 Trondheim
Hi again!
On 08. des. 2014 21:25, Ken Tobias wrote:
Thanks. Appreciated.
We are very interested in the OIDs you found. One problem we have
at the
moment is that we have no way of testing this in an exact matter as we have no 3850. I have asked around some and am expecting some feedback shortly. I
will
let you know what we discover.
Just to give you some feedback here - we didn't get any information from any sources that explained the behaviour you were experiencing. It seemed like a software update was the initiator of the problem and without access to the specific device type (Cisco 3850) and software we are running blind.
Another troubling trend is that later versions of Cisco software has stopped supporting SNMP for configuring devices and now requires NETCONF for such tasks. Thus we have as a task next year to implement NETCONF support in NAV so that we are still able to use the tools that configure devices.
PortAdmin is a popular tool, and we definitely plan to support it. It seems that there are a lot of changes going on now in especially Cisco software making it unstable, and we plan to work on that next year.
On Mon, 15 Dec 2014 11:21:00 +0100 John Magne Bredal john.m.bredal@uninett.no wrote:
Another troubling trend is that later versions of Cisco software has stopped supporting SNMP for configuring devices and now requires NETCONF for such tasks.
I'd like to point out that we only heard this about NX-OS software (Cisco Nexus products). We don't know yet that they have the same plans for IOS.
What would be interesting, Ken Tobias, is to know the two software versions of your working and your not-entirely-working 3850's.
Also, one of our network engineers was interested in seeing an example difference in output between `show vlan` and `show running` which demonstrates the "ephemeral" setting of VLANs you mentioned. He'd never heard of such a thing :-)
Below are some excerpts from one of the switches in question. Using the NAV "Configure Ports" button from the device page to get the entire list of ports, I set the name of g1/0/30 to "NAV Test" and the VLAN to 501. The same result is seen while using the "Configure Port" button from the specific port page.
"Before" shows that g1/0/30 is not in VLAN 501 and has no description using "show vlan id", "sh run int" and "sh int desc".
"After" shows that "sh vlan id" has g1/0/30 in vlan 501 and that "sh int desc" has the correct description while a "show run int" does not reflect these changes.
#sh ver Cisco IOS Software, IOS-XE Software, Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 03.06.01E RELEASE SOFTWARE (fc3)
*Before:* #sh vlan id 501 ... VLAN Name Status Ports ---- -------------------------------- --------- ------------------------------- 501 wla-adi1 active Po1 ...
#sh run int g1/0/30 ... interface GigabitEthernet1/0/30 switchport mode access power inline never spanning-tree portfast service-policy input Policy-3850-Input end
#sh int description | inc 30 Gi1/0/30 down down ...
*After:* #sh vlan id 501
VLAN Name Status Ports ---- -------------------------------- --------- ------------------------------- 501 wla-adi1 active Gi1/0/30, Po1
#sh run int g1/0/30 ... interface GigabitEthernet1/0/30 switchport mode access power inline never spanning-tree portfast service-policy input Policy-3850-Input end
#sh int description | inc 30 Gi1/0/30 admin down down NAV Test ...
Hope this helps. Please let me know if you would like additional output or diagnostics.
BTW, has anyone tried this on any other platform (not just 3850) using IOS XE 3.6.1E or something close?
On Tue, Dec 16, 2014 at 12:14 AM, Morten Brekkevold < morten.brekkevold@uninett.no> wrote:
On Mon, 15 Dec 2014 11:21:00 +0100 John Magne Bredal < john.m.bredal@uninett.no> wrote:
Another troubling trend is that later versions of Cisco software has stopped supporting SNMP for configuring devices and now requires NETCONF for such tasks.
I'd like to point out that we only heard this about NX-OS software (Cisco Nexus products). We don't know yet that they have the same plans for IOS.
What would be interesting, Ken Tobias, is to know the two software versions of your working and your not-entirely-working 3850's.
Also, one of our network engineers was interested in seeing an example difference in output between `show vlan` and `show running` which demonstrates the "ephemeral" setting of VLANs you mentioned. He'd never heard of such a thing :-)
-- Morten Brekkevold UNINETT
Update!
I updated a switch to IOS XE 3.7.0E. Now the VLANs are set correctly in the config file. So this appears to have been a Cisco bug.
However, the description still never gets set. The following error shows up in the apache2 log:
[Thu Dec 18 11:07:01 2014] [error] [Thu Dec 18 11:07:01 2014] [INFO] [pid=16607 nav.web.portadmin] ktobias: wladc-acc-sw-03:Gi2/0/30 - ifalias set to "NAV Test" [Thu Dec 18 11:07:05 2014] [error] [Thu Dec 18 11:07:05 2014] [ERROR] [pid=16607 nav.web.portadmin] Error doing write mem on wladc-acc-sw-03: Timed out waiting for SNMP response: Timeout
Another question. Is the "enable" behavior by design? As long as I have been using NAV (~1yr), any change to a port config VLAN or description results in the port being "shutdown". Even if the "enable" checkbox is selected. I have to then refresh the page to query the new status, check the "enable" box with is now not selected and save.
On Wed, Dec 17, 2014 at 3:53 PM, Ken Tobias kentobias@wcn.co wrote:
Below are some excerpts from one of the switches in question. Using the NAV "Configure Ports" button from the device page to get the entire list of ports, I set the name of g1/0/30 to "NAV Test" and the VLAN to 501. The same result is seen while using the "Configure Port" button from the specific port page.
"Before" shows that g1/0/30 is not in VLAN 501 and has no description using "show vlan id", "sh run int" and "sh int desc".
"After" shows that "sh vlan id" has g1/0/30 in vlan 501 and that "sh int desc" has the correct description while a "show run int" does not reflect these changes.
#sh ver Cisco IOS Software, IOS-XE Software, Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 03.06.01E RELEASE SOFTWARE (fc3)
*Before:* #sh vlan id 501 ... VLAN Name Status Ports
501 wla-adi1 active Po1 ...
#sh run int g1/0/30 ... interface GigabitEthernet1/0/30 switchport mode access power inline never spanning-tree portfast service-policy input Policy-3850-Input end
#sh int description | inc 30 Gi1/0/30 down down ...
*After:* #sh vlan id 501
VLAN Name Status Ports
501 wla-adi1 active Gi1/0/30, Po1
#sh run int g1/0/30 ... interface GigabitEthernet1/0/30 switchport mode access power inline never spanning-tree portfast service-policy input Policy-3850-Input end
#sh int description | inc 30 Gi1/0/30 admin down down NAV Test ...
Hope this helps. Please let me know if you would like additional output or diagnostics.
BTW, has anyone tried this on any other platform (not just 3850) using IOS XE 3.6.1E or something close?
On Tue, Dec 16, 2014 at 12:14 AM, Morten Brekkevold < morten.brekkevold@uninett.no> wrote:
On Mon, 15 Dec 2014 11:21:00 +0100 John Magne Bredal < john.m.bredal@uninett.no> wrote:
Another troubling trend is that later versions of Cisco software has stopped supporting SNMP for configuring devices and now requires NETCONF for such tasks.
I'd like to point out that we only heard this about NX-OS software (Cisco Nexus products). We don't know yet that they have the same plans for IOS.
What would be interesting, Ken Tobias, is to know the two software versions of your working and your not-entirely-working 3850's.
Also, one of our network engineers was interested in seeing an example difference in output between `show vlan` and `show running` which demonstrates the "ephemeral" setting of VLANs you mentioned. He'd never heard of such a thing :-)
-- Morten Brekkevold UNINETT
On 18. des. 2014 20:16, Ken Tobias wrote:
Update!
Thanks a lot for the exerpts! We will take a look at them.
I updated a switch to IOS XE 3.7.0E. Now the VLANs are set correctly in the config file. So this appears to have been a Cisco bug.
Very good!
However, the description still never gets set. The following error shows up in the apache2 log:
[Thu Dec 18 11:07:01 2014] [error] [Thu Dec 18 11:07:01 2014] [INFO] [pid=16607 nav.web.portadmin] ktobias: wladc-acc-sw-03:Gi2/0/30 - ifalias set to "NAV Test" [Thu Dec 18 11:07:05 2014] [error] [Thu Dec 18 11:07:05 2014] [ERROR] [pid=16607 nav.web.portadmin] Error doing write mem on wladc-acc-sw-03: Timed out waiting for SNMP response: Timeout
This tells me that the snmp-query that set the description actually succeeded but that the 'write mem' failed. If you say that the description was not set, then the 'write mem' has in some way messed with that. That is very strange. I am tempted to move the 'write mem' functionality to a config option, so that you can turn it off. Then we can see how many problems are caused by it.
Another question. Is the "enable" behavior by design? As long as I have been using NAV (~1yr), any change to a port config VLAN or description results in the port being "shutdown". Even if the "enable" checkbox is selected. I have to then refresh the page to query the new status, check the "enable" box with is now not selected and save.
This should not happen. If and only if you have changed vlans the interface is "restarted", that is set adminitratively down and up (with a 5 second waiting time). This is so that any computers behind the interface detects that the network is down and tries to aquire a new IP-address.
I have added some more debug-statements that will come along in the next NAV release. Hopefully this will help us understand more.
On Fri, 19 Dec 2014 10:30:35 +0100 John Magne Bredal john.m.bredal@uninett.no wrote:
Another question. Is the "enable" behavior by design? As long as I have been using NAV (~1yr), any change to a port config VLAN or description results in the port being "shutdown". Even if the "enable" checkbox is selected. I have to then refresh the page to query the new status, check the "enable" box with is now not selected and save.
This should not happen. If and only if you have changed vlans the interface is "restarted", that is set adminitratively down and up (with a 5 second waiting time). This is so that any computers behind the interface detects that the network is down and tries to aquire a new IP-address.
I have added some more debug-statements that will come along in the next NAV release. Hopefully this will help us understand more.
Ken & John-Magne,
how are those debug statements working out, is there enough information to get to the bottom if this issue now?
-
John Magne Bredal -
Ken Tobias -
kentobias＠wcn.co
-
Morten Brekkevold