Add support for configure private, without DB locking

[omershtivi]

No description provided.

[coveralls]

Coverage decreased (-0.07%) to 78.895% when pulling e480a1f on omershtivi:juniper_configure_private into c5e5f60 on napalm-automation:develop.

[bewing]

It looks like the config object supports using the private database, but only if called with a context manager:

http://junos-pyez.readthedocs.io/en/2.1.8/jnpr.junos.utils.html#module-jnpr.junos.utils.config

Do we know why the context manager is required? Can we leverage this instead of sending the rpc command directly?

[bewing]

From https://www.juniper.net/documentation/en_US/junos-pyez/topics/reference/general/junos-pyez-configuration-process-and-data-formats.html

Starting in Junos PyEZ Release 2.0, you can specify the configuration mode to use when making structured or unstructured configuration changes. To make configuration changes in a mode other than the default configuration mode, which updates the shared configuration database, you must create the Config or Table object using a context manager and include the mode parameter in the argument list. Supported modes include private, exclusive, dynamic, and batch.

When you specify one of these modes, the context manager handles opening and locking and closing and unlocking the database. This ensures that you do not unintentionally leave the database in a locked state. Thus, you only need to call the load() and commit() methods to configure the device.

All the context manager does is handle issuing lock/unlock operations for the specific config mode -- should we just call them as part of JunOSDriver._lock()/_unlock(), and by default use mode="exclusive", but allow other modes?

[omershtivi]

Hi,
It's not sending command directly to RPC but instead change the device RPC configuration to be private.
It's because we load the "config" module into the device and not vice versa.
https://forums.juniper.net/t5/Junos-Automation-Scripting/How-to-set-PyEz-to-quot-configure-private-quot-mode/td-p/283862

And I think the common modes are private, exclusive, and configure. So private and exclusive are covered.

[mirceaulinic]

This looks good to me on the surface, though I'd like to test it before getting this in - mainly to ensure that the config DB is properly unlocked on discard / rollback / disconnect.

[omershtivi]

So how can we perform the checks?

[mirceaulinic]

@omershtivi: Unfortunately this is a type of thing that cannot really be part of a test suite to be trusted enough, so I am going to test it effectively on a real box or VM in various scenarios. I hope to be able to get to this later today or next week. Cheers.

[omershtivi]

@mirceaulinic I've tested the above (discarding, rollback, and disconnect) and everything worked fine.
Let me know if you have any issues, I'm staring to use it with nornir

[omershtivi]

Hi,
Can we merge it?

[ktbyers]

@omershtivi Can you give an overview of why you need this feature? What the use case would be? I have a sense of it, but I want to make sure my assumptions are not wrong:

Main concerns here are:

1. Can we adequate test this across time? It probably would need to be added into the napalm_custom_test library so that automatic testing this regularly.
2. The other concern I had is what happens with respect to NAPALM and configure private (in the case when someone else makes a commit i.e. someone non-NAPALM makes a commit to the configuration).

I guess the other question is whether we (NAPALM) should even do this or whether this should just be outside of napalm i.e. we have a separate PR (#881) which allows you to control locking outside of napalm so if you want to do that, we provide a way (but then it is up to that system to ensure the unexpected error states are handled).

I am probably inclined towards this last position, but am open to different arguments on this.

[omershtivi]

Hi @ktbyers,
Because we have more then one automation script and we need to avoid locking the DB from other systems, or commiting something that we didn't intended, and when we crash not leaving anything in the candidate/locking the DB.
We have some more reasons.

1. I have no problems adding tests, but it will take me some time.
2. Can you please elaborate?

DB locking is just a portion of it, really need the private candidate file to avoid overlapping.

[mirceaulinic]

Hi @omershtivi - there seem to be some conflicts, would you be able to resolve them?

[mirceaulinic]

👋 @omershtivi - I've opened #1169 to revisit this, as I couldn't find the branch for this PR on your NAPALM fork. I've simply copied your changes, let me know if you'd prefer to continue working on this PR, re-submit another one, or go ahead with mine. 😉

[mirceaulinic]

Closing this in the favour of #1169. Thanks for your contribution @omershtivi, and sorry for not looking into this earlier as promised! 🙏