python-openstackclient

'server list --sort-column Networks' exits with 'list index out of range'

Bug #2152566 reported by Matthias Harzer
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cliff
Fix Committed
Undecided
Matthias Harzer
python-openstackclient
In Progress
Undecided
Udayendu Kar

Bug Description

When trying to sort servers by their networks using

openstack server list --sort-column Networks

the command exits with a message 'list index out of range' (no table being printed). Other sort columns work as expected, only the 'Networks' column seem to cause this behavior.

When using the `--debug` flag, it reveals that this issue seems to lay in the 'cliff' package, which incorrectly handles a prior error. During sorting, cliff tries to compare values from the 'Networks' column, which seem to be incomparable. This raises an error. In case of such failure, ordering should be skipped (ordering is best effort), but during recovery, cliff fails again (this time with the out of range error) while trying to log the compare-error [1].
The index out of range error is due to the wrong array being used ('parsed_args.sort_columns' instead of 'column_names' which the index refers to) during construction of a nice log message [2].

Although fixing this in cliff would prevent this error, it would still not sort the columns as requested, which may be another issue.

[1] https://opendev.org/openstack/cliff/src/commit/9e12d0980849f17c4fa691138f478f406d7488f3/cliff/lister.py#L106-L124
[2] https://opendev.org/openstack/cliff/src/commit/9e12d0980849f17c4fa691138f478f406d7488f3/cliff/lister.py#L123

Joonseok Kim (joonseok-kim)
Changed in python-openstackclient:
assignee: nobody → Joonseok Kim (lylekim)
assignee: Joonseok Kim (lylekim) → nobody
Revision history for this message
Udayendu Kar (uskar88) wrote :
Download full text (3.9 KiB)

Hi Matthias, I was able to reproduce the issue and given a patch: https://review.opendev.org/c/openstack/cliff/+/995202

Now its able to sort the Networks field properly. Please try once and let me know. The issue was with cliff package.

With this fix, the output is going to be like:

(kolla-venv) root@uskar-dev-node:~# openstack server list
+--------------------------------------+----------+--------+------------------------+--------+---------+
| ID | Name | Status | Networks | Image | Flavor |
 +--------------------------------------+----------+--------+------------------------+--------+---------+
| f40c37eb-dfe3-48ce-b5fa-9974f167d03c | server-5 | ACTIVE | demo-net=10.0.0.144 | cirros | m1.tiny |
| 56c7a2f4-7c15-42e9-b73f-1d1db2a7c977 | server-4 | ACTIVE | network4=192.168.4.222 | cirros | m1.tiny |
| 967b89ed-a010-4334-84d0-761b8f614d92 | server-3 | ACTIVE | network3=192.168.3.128 | cirros | m1.tiny |
| 235c0e33-26a7-4cac-8d09-920d076c731f | server-1 | ACTIVE | network1=192.168.1.254 | cirros | m1.tiny |
 +--------------------------------------+----------+--------+------------------------+--------+---------+
(kolla-venv) root@uskar-dev-node:~# openstack server list --sort-column ID
+--------------------------------------+----------+--------+------------------------+--------+---------+
| ID | Name | Status | Networks | Image | Flavor |
 +--------------------------------------+----------+--------+------------------------+--------+---------+
| 235c0e33-26a7-4cac-8d09-920d076c731f | server-1 | ACTIVE | network1=192.168.1.254 | cirros | m1.tiny |
| 56c7a2f4-7c15-42e9-b73f-1d1db2a7c977 | server-4 | ACTIVE | network4=192.168.4.222 | cirros | m1.tiny |
| 967b89ed-a010-4334-84d0-761b8f614d92 | server-3 | ACTIVE | network3=192.168.3.128 | cirros | m1.tiny |
| f40c37eb-dfe3-48ce-b5fa-9974f167d03c | server-5 | ACTIVE | demo-net=10.0.0.144 | cirros | m1.tiny |
 +--------------------------------------+----------+--------+------------------------+--------+---------+
(kolla-venv) root@uskar-dev-node:~# openstack server list --sort-column Name
+--------------------------------------+----------+--------+------------------------+--------+---------+
| ID | Name | Status | Networks | Image | Flavor |
 +--------------------------------------+----------+--------+------------------------+--------+---------+
| 235c0e33-26a7-4cac-8d09-920d076c731f | server-1 | ACTIVE | network1=192.168.1.254 | cirros | m1.tiny |
| 967b89ed-a010-4334-84d0-761b8f614d92 | server-3 | ACTIVE | network3=192.168.3.128 | cirros | m1.tiny |
| 56c7a2f4-7c15-42e9-b73f-1d1db2a7c977 | server-4 | ACTIVE | network4=192.168.4.222 | cirros | m1.tiny |
| f40c37eb-dfe3-48ce-b5fa-9974f167d03c | server-5 | ACTIVE | demo-net=10.0.0.144 | cirros | m1.tiny |
 +--------------------------------------+----------+--------+------------------------+--------+---------+
(kolla-venv) root@uskar-dev-node:~# openstack server list --sort-column Networks
+--------------------------------------+----------+--------+------------------------+--------+--------...

Read more...

Changed in python-openstackclient:
assignee: nobody → Udayendu Kar (uskar88)
status: New → In Progress
Udayendu Kar (uskar88)
affects: python-openstackclient → cliff
affects: cliff → python-cliff
Udayendu Kar (uskar88)
Changed in python-cliff:
status: In Progress → Fix Committed
Udayendu Kar (uskar88)
Changed in python-cliff:
status: Fix Committed → In Progress
Revision history for this message
Udayendu Kar (uskar88) wrote :

Instead of fixing it in cliff, lets fix it in the python-openstack client itself. No need to make the cliff logic more complex, instead pass the correct value to it to sort the Networks column.

I will make a patch and submit to python-openstackclient.

affects: python-cliff → python-openstackclient
Revision history for this message
Udayendu Kar (uskar88) wrote :

Added a new patch to python-openstackclient: https://review.opendev.org/c/openstack/python-openstackclient/+/999454

Revision history for this message
Udayendu Kar (uskar88) wrote :

Without the above fix in python-openstackclient, the error was like:

---
Could not sort on field 'Networks'; unsortable types
---

But the sorting was not working, just the TypeError was masked.

Revision history for this message
Matthias Harzer (matthiasharzer) wrote :

The error described in my original bug-report is an error of the cliff-package, since the cliff-package did not recover from an already failed sorting attempt. According to cliff, sorting is best effort [1]. This is a cliff-issue and should be fixed by [2].

What you described is the logical (and IMO required) consequence in the python-openstackclient, which didn't use the cliff package correctly in the first place. I think both packages need to be fixed. cliff should never crash, especially not during an attempt to log a message and the openstackclient should provide a way to sort Networks.

[1] https://opendev.org/openstack/cliff/src/commit/89efc71e3660e0f5b5fc56fa455890950c205401/cliff/lister.py#L120
[2] https://review.opendev.org/c/openstack/cliff/+/990593

Revision history for this message
Udayendu Kar (uskar88) wrote (last edit ):

Yes, true. Your code is having the fix for the TypeError in Cliff and the fix I submitted is related to the actual sorting issue happening due to the data type in python-openstackclient. Both are relevant in order to sort the Networks column.

I am going to add the cliff package also into this bug.

Revision history for this message
Udayendu Kar (uskar88) wrote :

Hi Matthias, For the Cliff issue could you please assign that to yourself and close the bug.

Matthias Harzer (matthiasharzer)
Changed in python-cliff:
assignee: nobody → Matthias Harzer (matthiasharzer)
status: New → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.