Tag Archives: openstack

[Install OpenStack] Sync keystone database error: keystone DBConnectionError: (pymysql.err.OperationalError) (2003, “Can’t connect to MySQL server on ‘controller’ ([Errno 111] Connection refused)”)

Execute the command su -s /bin/sh -c “keystone-manage db_sync” keystone keeps failing to execute, check /var/log/keystone/keystone.log and find the following error:
2021-09-27 09:05:18.381 185184 ERROR keystone Traceback (most recent call last):
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/bin/keystone-manage”, line 10, in <module>
2021-09-27 09:05:18.381 185184 ERROR keystone sys.exit(main())
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/lib/python2.7/site-packages/keystone/cmd/manage.py”, line 44, in main
2021-09-27 09:05:18.381 185184 ERROR keystone cli.main(argv=sys.argv, config_files=config_files)
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/lib/python2.7/site-packages/keystone/cmd/cli.py”, line 1312, in main
2021-09-27 09:05:18.381 185184 ERROR keystone CONF.command.cmd_class.main()
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/lib/python2.7/site-packages/keystone/cmd/cli.py”, line 467, in main
2021-09-27 09:05:18.381 185184 ERROR keystone CONF.command.version)
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/lib/python2.7/site-packages/keystone/common/sql/upgrades.py”, line 251, in offline_sync_database_t o_version
2021-09-27 09:05:18.381 185184 ERROR keystone expand_schema()
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/lib/python2.7/site-packages/keystone/common/sql/upgrades.py”, line 271, in expand_schema
2021-09-27 09:05:18.381 185184 ERROR keystone _sync_common_repo(version=None)
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/lib/python2.7/site-packages/keystone/common/sql/upgrades.py”, line 167, in _sync_common_repo
2021-09-27 09:05:18.381 185184 ERROR keystone with sql.session_for_write() as session:
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/lib64/python2.7/contextlib.py”, line 17, in __enter__
2021-09-27 09:05:18.381 185184 ERROR keystone return self.gen.next()
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py”, line 944, in _transaction_scope
2021-09-27 09:05:18.381 185184 ERROR keystone allow_async=self._allow_async) as resource:
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/lib64/python2.7/contextlib.py”, line 17, in __enter__
2021-09-27 09:05:18.381 185184 ERROR keystone return self.gen.next()
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py”, line 558, in _session
2021-09-27 09:05:18.381 185184 ERROR keystone bind=self.connection, mode=self.mode)
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py”, line 317, in _create_session
2021-09-27 09:05:18.381 185184 ERROR keystone self._start()
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py”, line 403, in _start
2021-09-27 09:05:18.381 185184 ERROR keystone engine_args, maker_args)
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py”, line 427, in _setup_for_connectio n
2021-09-27 09:05:18.381 185184 ERROR keystone sql_connection=sql_connection, **engine_kwargs)
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py”, line 155, in create_engine
2021-09-27 09:05:18.381 185184 ERROR keystone test_conn = _test_connection(engine, max_retries, retry_interval)
2021-09-27 09:05:18.381 185184 ERROR keystone File “/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py”, line 339, in _test_connection
2021-09-27 09:05:18.381 185184 ERROR keystone six.reraise(type(de_ref), de_ref)
2021-09-27 09:05:18.381 185184 ERROR keystone File “<string>”, line 2, in reraise
2021-09-27 09:05:18.381 185184 ERROR keystone DBConnectionError: (pymysql.err.OperationalError) (2003, “Can’t connect to MySQL server on ‘controller’ ([Errno 111] Connection refused)”)
2021-09-27 09:05:18.381 185184 ERROR keystone
2021-09-27 10:09:09.147 165810 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. 10 attempts left.
2021-09-27 10:09:19.158 165810 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. 9 attempts left.
2021-09-27 10:09:29.169 165810 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. 8 attempts left.
2021-09-27 10:09:39.181 165810 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. 7 attempts left.
2021-09-27 10:09:49.193 165810 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. 6 attempts left.
2021-09-27 10:09:59.204 165810 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. 5 attempts left.
2021-09-27 10:10:09.216 165810 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. 4 attempts left.
2021-09-27 10:10:19.228 165810 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. 3 attempts left.
2021-09-27 10:10:29.240 165810 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. 2 attempts left.
2021-09-27 10:10:39.251 165810 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. 1 attempts left.
2021-09-27 10:10:49.255 165810 CRITICAL keystone [-] DBConnectionError: (pymysql.err.OperationalError) (2003, “Can’t connect to MySQL server on ‘controller’ ([Errno 111] Connection refused)”)
Type the log at the end of the error report
Looking at the keystone logs again, you can see that the execution is now as follows
After searching the internet the format should be
engine(“mysql+pymysql://{user}:{password}@{host}:{port}/ajx?charset=utf8”.format(**config))
It’s easy to see that the port is missing

Modify /etc/keystone/keystone.conf to add the port


Execute the sync command again

Openstack Create instance error: Failed to allocate the network(s) with error No fixed IP addresses available for network

Openstack Create instance error: Failed to allocate the network(s) with error No fixed IP addresses available for network

 

The log indicates that sending HDR request to TGT fails, so there is a problem with tgtd service [tgtd iSCSI target daemon]. Tgtd iSCSI target daemon means that tgtd is the background service of iSCSI target, that is, the daemon. Execute the command systemctl status tgtd and find that the service is in the disabled state, so start the service systemctl start tgtd to solve the problem

Openstack creates a new instance and solves various error reports

Recently, I installed openstack, zero basic installation, and referred to many online tutorials

The consequence of eating a hundred meals is that there are a lot of unknown problems… Openstack installation is complex. Many configuration files are incorrectly configured in one place, which may lead to the unavailability of later functions

This document only records a series of errors and troubleshooting process encountered when starting the instance after installation

BUG 1: No valid host

Error reporting

No valid host was found. There are not enough hosts available.

Solution

Network node execution

[root@openstack-controller-dev ~]# vim /etc/sysctl.conf 

Add the following:

net.ipv4.ip_forward=1
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0

Verify whether it is effective

[root@openstack-controller-dev ~]# sysctl -p
net.ipv4.ip_forward = 1
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0

 

BUG 2: Unable to convert image to raw

error

69ad3af8-3253-4a35-a6f1-ee5bcd1e37f2 aborted: Image 8f9cf451-764e-4219-ba0b-2edb93a9e63e is unacceptable: Unable to convert image to raw: Image /var/lib/nova/instances/_base/9b2bd71aef84e92d7147d0eb3697710afd403a4a.part is unacceptable: Unable to convert image to raw: Unexpected error while running command.
2019-11-15 01:12:18.776 162849 ERROR nova.compute.manager [instance: 69ad3af8-3253-4a35-a6f1-ee5bcd1e37f2] Command: qemu-img convert -O raw -f qcow2 /var/lib/nova/instances/_base/9b2bd71aef84e92d7147d0eb3697710afd403a4a.part /var/lib/nova/instances/_base/9b2bd71aef84e92d7147d0eb3697710afd403a4a.converted
2019-11-15 01:12:18.776 162849 ERROR nova.compute.manager [instance: 69ad3af8-3253-4a35-a6f1-ee5bcd1e37f2] Exit code: 1
2019-11-15 01:12:18.776 162849 ERROR nova.compute.manager [instance: 69ad3af8-3253-4a35-a6f1-ee5bcd1e37f2] Stdout: u''
2019-11-15 01:12:18.776 162849 ERROR nova.compute.manager [instance: 69ad3af8-3253-4a35-a6f1-ee5bcd1e37f2] Stderr: u'qemu-img: error while reading sector 17280: Input/output error\n'

Troubleshooting process

After developing for Baidu, I found the answer on openstack’s official website QA: the image upload is incomplete

 

Reference link: image is unacceptable: unable to convert image to raw

So upload again, and find new problems after uploading

BUG 3: CPU feature spec-ctrl not found

Error reporting

internal error: process exited while connecting to monitor: 2019-11-15T09:42:49.789389Z qemu-kvm: CPU feature spec-ctrl not found

Analysis and troubleshooting

After Baidu checked many articles, I found that the following one is very reasonable

Reference link: can openstack swallow the skylake hot dog

When checking the Nova compute log of the corresponding computing node, it is found that there is an error message in the log

qemu-kvm: CPU feature spec-ctrl not found

The log conveys several messages:

Libvirt’s get host CPU instruction set list contains this feature

QEMU KVM does not support the CPU feature spec Ctrl

Solution

Upgrade libvirt and QEMU to fit the new CPU

Modify the instruction set list obtained by the compute node libvirt_Map.xml to mask CPU features that are not supported by virtualization components. Backup CPU_Map. XML file, and then modify/usr/share/libvirt/CPU_Map.xml, and delete the spec Ctrl related features

 

Finally, restart the libvirt service to take effect

Considering that the upgrade of libvirt and QEMU has a great impact and may affect some logic and functions after the upgrade, the second solution is recommended

Configuration like this

<model name='Haswell-noTSX-IBRS'>
  <model name='Haswell-noTSX'/>
  <feature name='spec-ctrl'/>
</model>

Change to

<model name='Haswell-noTSX-IBRS'>
  <model name='Haswell-noTSX'/>
</model>

Delete such directly

<feature name='spec-ctrl'>
  <cpuid eax_in='0x07' ecx_in='0x00' edx='0x04000000'/>
</feature>

Restart libvirt

[root@openstack-compute-dev ~]# systemctl restart libvirtd.service

After the above operations are completed, create a new instance again, and the following errors may be reported:

internal error: process exited while connecting to monitor: 2019-11-15T11:02:14.259953Z qemu-kvm: CPU feature stibp not found

Just perform the same operation as above for spec Ctrl

 

Error accessing horizon after installing dashboard in openstack end of script output before headers: django.wsgi

Add the following sentence to the configuration file to solve the problem

/etc/apache2/conf-available/openstack-dashboard.conf

WSGIApplicationGroup %{GLOBAL}

[Thu Jan 04 11:32:23.287776 2018] [mpm_event:notice] [pid 82455:tid 139795941345152] AH00489: Apache/2.4.7 (Ubuntu) mod_wsgi/3.4 Python/2.7.6 configured — resuming normal operations
[Thu Jan 04 11:32:23.287863 2018] [core:notice] [pid 82455:tid 139795941345152] AH00094: Command line: ‘/usr/sbin/apache2’
[Thu Jan 04 03:32:48.606837 2018] [:error] [pid 82458:tid 139795793630976] Could not process panel theme_preview: Dashboard with slug “developer” is not registered.
[Thu Jan 04 03:32:52.373952 2018] [:error] [pid 82459:tid 139795835594496] Could not process panel theme_preview: Dashboard with slug “developer” is not registered.
[Thu Jan 04 03:32:54.697505 2018] [:error] [pid 82460:tid 139795835594496] Could not process panel theme_preview: Dashboard with slug “developer” is not registered.
[Thu Jan 04 03:33:29.355404 2018] [:error] [pid 82460:tid 139795760060160] Login successful for user “admin”.

Configure apache with -l 127.0.0.1 and the configuration ip in Apache is not the same modify has been solved