Improve some sqlite3 errors

[serhiy-storchaka]

BPO	44859
Nosy	@berkerpeksag, @serhiy-storchaka, @JelleZijlstra, @erlend-aasland
PRs	bpo-44859: Improve error handling in sqlite3 and change some errors #27654 bpo-44859: Raise more accurate exceptions in sqlite3 #27695

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-08-07.14:14:16.688>
labels = ['extension-modules', '3.11']
title = 'Improve some sqlite3 errors'
updated_at = <Date 2022-03-17.05:58:28.684>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2022-03-17.05:58:28.684>
actor = 'JelleZijlstra'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules']
creation = <Date 2021-08-07.14:14:16.688>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 44859
keywords = ['patch']
message_count = 8.0
messages = ['399183', '399185', '399187', '399211', '399241', '399289', '399441', '415388']
nosy_count = 4.0
nosy_names = ['berker.peksag', 'serhiy.storchaka', 'JelleZijlstra', 'erlendaasland']
pr_nums = ['27654', '27695']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue44859'
versions = ['Python 3.11']

[serhiy-storchaka]

• MemoryError is now raised instead of sqlite3.Warning when memory is not enough for encoding statement to UTF-8 in Connection.__call__() and Cursor.execute().
• UnicodEncodeError is now raised instead of sqlite3.Warning when statement contains surrogate characters in Connection.__call__() and Cursor.execute().
• TypeError is now raised instead of ValueError for non-string script in Cursor.execute().
• ValueError is now raised for script containing NUL instead of truncating it in Cursor.execute().
• Correctly handled exceptions raised when getting boolean value of the result of the progress handler to bool.

Also added many tests which cover different exceptional cases.

[serhiy-storchaka]

There is also a problem with the sqlite3.InterfaceError raised when bind parameters with message "Error binding parameter XXX - probably unsupported type." It is raised not only for unsupported types, but:

• For too large integers (outside of the range of signed 64-bit integers).
• For too large strings and blobs with size <= INT_MAX but larger than the SQLite limit (1_000_000_000 by default). But OverflowError is raised for size >= INT_MAX.
• When memory error or any other error occurred inside SQLite.

[serhiy-storchaka]

Entries 3 and 4 are about Cursor.executescript(), not Cursor.execute().

[serhiy-storchaka]

New changeset 0eec627 by Serhiy Storchaka in branch 'main':
bpo-44859: Improve error handling in sqlite3 and and raise more accurate exceptions. (GH-27654)
0eec627

[erlend-aasland]

• sqlite3.ProgrammingError is raised for SQLITE_MISUSE. IMO, sqlite3.InterfaceError, is more appropriate.
• In case of too large integer values, OverflowError is raised on bind, but sqlite3.DataError is raised on UDF return. Using sqlite3.DataError in both cases would be more aligned with PEP-249.
• Too long strings/blobs raise OverflowError. Consider raising sqlite3.DataError instead.
• For non-contiguous (blob) buffers, we overwrite the BufferError with ValueError. Instead of overwriting BufferError, we should consider chaining with sqlite3.DataError.
• Consider raising sqlite3.Warning when nan is implicitly converted to NULL by SQLite
• In pysqlite_statement_create(), consider raising sqlite3.ProgrammingError iso. sqlite3.Warning when more than one statement is executed
• In pysqlite_statement_create(), consider raising sqlite3.DataError iso. ValueError for queries with null characters

[erlend-aasland]

Also, for SQLite 3.7.17 and above, we should raise sqlite3.Warning for error codes SQLITE_NOTICE and SQLITE_WARNING.

[erlend-aasland]

Also, for SQLite 3.7.17 and above, we should raise sqlite3.Warning for error codes SQLITE_NOTICE and SQLITE_WARNING.

Er, forget that. Neither are returned by any SQLite C interface, at the moment. They can (currently) only be used as the first argument to sqlite3_log().

[JelleZijlstra]

New changeset 4674fd4 by Erlend Egeberg Aasland in branch 'main':
bpo-44859: Raise more accurate exceptions in sqlite3 (GH-27695)
4674fd4

[erlend-aasland]

The following PRs have improved this issue:

• bpo-44859: Improve error handling in sqlite3 and change some errors #27654
• bpo-44859: Raise more accurate exceptions in sqlite3 #27695
• gh-69093: improve sqlite3.Connection.blobopen() error handling #91571
• gh-89022: Improve sqlite3 exceptions related to binding params and API misuse #91572

Suggesting to close this. If other inconsistencies are discovered, we can open a new issue.

[erlend-aasland]

Thanks for bringing this up and pushing it, Serhiy.

[erlend-aasland]

There is one inconsistency remaining:

sqlite3.Cursor.execute raise sqlite3.Warning instead of sqlite3.ProgrammingError if multiple SQL statements are supplied. IMO, we should fix this.

Actually, it is not raised in the execute method implementation; it is raised during statement creation in statement.c.

Forget it; I already fixed this ☕ ☕ ☕