Repository navigation
Closed
Labels
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/rhettinger'
closed_at = <Date 2017-09-24.23:15:12.135>
created_at = <Date 2016-11-20.16:51:59.575>
labels = ['3.7', 'expert-argument-clinic']
title = 'Argument Clinic for bisect.bisect_left'
updated_at = <Date 2017-09-25.02:51:14.535>
user = 'https://github.com/JulienPalard'bugs.python.org fields:
activity = <Date 2017-09-25.02:51:14.535>
actor = 'rhettinger'
assignee = 'rhettinger'
closed = True
closed_date = <Date 2017-09-24.23:15:12.135>
closer = 'rhettinger'
components = ['Argument Clinic']
creation = <Date 2016-11-20.16:51:59.575>
creator = 'mdk'
dependencies = []
files = ['45562', '45563', '45568', '45569', '45570', '45571', '45572', '45626', '45628', '45642', '45650', '45693', '45832', '45836']
hgrepos = []
issue_num = 28754
keywords = ['patch']
message_count = 54.0
messages = ['281282', '281283', '281285', '281299', '281302', '281303', '281652', '281669', '281685', '281712', '281713', '281717', '281734', '281761', '281762', '281763', '281764', '281765', '281768', '281772', '281779', '281791', '281792', '281832', '281834', '281878', '281880', '281888', '281892', '281899', '281901', '281926', '282028', '282030', '282038', '282041', '282049', '282063', '282308', '282350', '282842', '282854', '282864', '282875', '282877', '282882', '282888', '282889', '282891', '283100', '291160', '291162', '302894', '302908']
nosy_count = 7.0
nosy_names = ['rhettinger', 'vstinner', 'larry', 'skrah', 'martin.panter', 'serhiy.storchaka', 'mdk']
pr_nums = ['177']
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue28754'
versions = ['Python 3.7']
Activity
JulienPalard commentedon Nov 20, 2016
Today I read https://docs.python.org/3.6/howto/clinic.html so I tried one: bisect.bisect_left.
I was unable to do
bisect_right, as it's an "alias" forbisect, and there's a unit-test checkingself.assertEqual(self.module.bisect, self.module.bisect_right). Any hint welcome.larryhastings commentedon Nov 20, 2016
There's special syntax to handle aliases. From comments in clinic.py:
Sorry the docs are out of date--patches welcome. And speaking of patches, thanks for the patch! I'll review it when you say it's ready.
larryhastings commentedon Nov 20, 2016
Oh, and, if the code literally asserts they're the same function, that's just a sanity check based on the implementation. You could preserve that if you care to, or you could just write a new function and remove the assertion. Do what you think is best, then post your patch to the tracker and we'll all argue about it forever, how's that sound?
JulienPalard commentedon Nov 20, 2016
I searched an occurrence of what I'm describing which is already using clinic and there is, at least, one in Modules/binascii.c line 1090:
TL;DR: The idea is to use the
modulename.fnname [as c_basename] = modulename.existing_fn_nameclinic syntax, drop a "This function is also available as …" in the docstring, and check if they return the same thing in the tests.This way looks good to me, it uses the dedicated syntax which looks right, and I'll drop the
self.assertEqual(self.module.bisect, self.module.bisect_right)which is an implementation detail, replacing it by tests checking that both have the same behavior.JulienPalard commentedon Nov 20, 2016
Here it is for the whole bisect module.
I separated my work in commits, but I'm not sure how rietveld will eat that as they'll have unknown references, so I'll probably also upload a single patch with a known reference.
JulienPalard commentedon Nov 20, 2016
The whole diff is reviewable in
bpo-28754-3.diff.vstinner commentedon Nov 24, 2016
Hum, I dislike your changes on test_bisect.py: I don't see why Argument Clinic would have to modify tests!?
I created issue bpo-28792: "bisect: implement aliases in Python, remove C aliases".
JulienPalard commentedon Nov 24, 2016
New simplier patch thanks to https://bugs.python.org/issue28792. Also corrected docstrings.
Also ran a micro-benchmark of
bisect.bisect(foo, "c")withfoo = list("abcdef"):Median +- std dev: [before] 434 ns +- 17 ns -> [after] 369 ns +- 22 ns: 1.18x faster
FTR: ./python -m perf timeit -s 'import bisect; foo = list("abdef")' 'bisect.bisect(foo, "c")' -o after.json --inherit=PYTHONPATH --affinity=2,3
vstinner commentedon Nov 25, 2016
bpo-28754-4.diff does 3 things:
bpo-28754-5.diff doesn't touch tests, but still does 2 things. Sadly, I dislike the changes on docstrings because it makes the C docstring and the Python docstring inconsistent.
I suggest to copy/paste docstring between C and Python code.
By the way, why do we have a docstring different from Doc/library/bisect.rst? Maybe we can simply use the same doc everywhere, but just add reST formatting in Doc/library/bisect.rst?
Julien: I suggest to first restrict your change to Argument Clinic, and later write a new patch to update docstrings. I prefer to only do one thing per commit.
vadmium commentedon Nov 25, 2016
Victor, what changes to the doc strings are you talking about? Adding descriptions of the parameters, maybe? Other than that they look very similar to me.
I haven’t looked closely, but the Python doc strings have been updated at least once more than the C doc strings: r54666 and bpo-1602378. So if you are copying anything, it might be better to copy from Py to C. But I would do that separately.
The doc strings and reference documentation just seem to have been added independently: ce2d120c3903 (main documentation) vs 67b3ac439f64 (doc strings)
vstinner commentedon Nov 25, 2016
Martin: "Victor, what changes to the doc strings are you talking about?"
See attached check_bisect_doc.py script.
Before:
---
=== bisect_right ===
bisect_right(a, x[, lo[, hi]]) -> index
Return the index where to insert item x in list a, assuming a is sorted.
The return value i is such that all e in a[:i] have e <= x, and all e in
a[i:] have e > x. So if x already appears in the list, i points just
beyond the rightmost x already there
Optional args lo (default 0) and hi (default len(a)) bound the
slice of a to be searched.
=== insort_right ===
Insert item x in list a, and keep it sorted assuming a is sorted.
If x is already in a, insert it to the right of the rightmost x.
Optional args lo (default 0) and hi (default len(a)) bound the
slice of a to be searched.
=== bisect_left ===
bisect_left(a, x[, lo[, hi]]) -> index
Return the index where to insert item x in list a, assuming a is sorted.
The return value i is such that all e in a[:i] have e < x, and all e in
a[i:] have e >= x. So if x already appears in the list, i points just
before the leftmost x already there.
Optional args lo (default 0) and hi (default len(a)) bound the
slice of a to be searched.
=== insort_left ===
Insert item x in list a, and keep it sorted assuming a is sorted.
If x is already in a, insert it to the left of the leftmost x.
Optional args lo (default 0) and hi (default len(a)) bound the
slice of a to be searched.
---
After:
---
=== bisect_right ===
Return the index where to insert item x in list a, assuming a is sorted.
a
The list in which x is to be inserted.
x
The value to be inserted.
lo
Lower bound, defaults to 0.
hi
Upper bound, defaults to -1 (meaning len(a)).
The return value i is such that all e in a[:i] have e <= x, and all e in
a[i:] have e > x. So if x already appears in the list, i points just
beyond the rightmost x already there.
Optional args lo and hi bound the slice of a to be searched.
=== insort_right ===
Insert item x in list a, and keep it sorted assuming a is sorted.
a
The list in which x will be inserted.
x
The value to insert.
lo
Lower bound, defaults to 0.
hi
Upper bound, defaults to -1 (meaning len(a)).
If x is already in a, insert it to the right of the rightmost x.
Optional args lo and hi bound the slice of a to be searched.
=== bisect_left ===
Return the index where to insert item x in list a, assuming a is sorted.
a
The list in which x is to be inserted.
x
The value to be inserted.
lo
Lower bound, defaults to 0.
hi
Upper bound, defaults to -1 (meaning len(a)).
The return value i is such that all e in a[:i] have e < x, and all e in
a[i:] have e >= x. So if x already appears in the list, i points just
before the leftmost x already there.
Optional args lo and hi bound the slice of a to be searched.
=== insort_left ===
Insert item x in list a, and keep it sorted assuming a is sorted.
a
The list in which x will be inserted.
x
The value to insert.
lo
Lower bound, defaults to 0.
hi
Upper bound, defaults to -1 (meaning len(a)).
If x is already in a, insert it to the left of the leftmost x.
Optional args lo and hi bound the slice of a to be searched.
---
The output is different. I suggest to not touch the docstring in the patch upgrading _bisect.c to Argument Clinic to ease the review.
vadmium commentedon Nov 25, 2016
Taking the first function, bisect_right(), as an example, I see these differences:
bisect_right(a, x[, lo[, hi]]) -> index
This signature is removed. I think removing it is reasonable, because pydoc can extract the proper signature from the Arg Clinic metadata.
Additional descriptions of each parameter. I tend to think these are redundant with the main text, so agree with removing them from the patch now.
Addition of full stop (.) at end of first paragraph. I suggested this as a minor cleanup, but it could be fixed later if you prefer.
Removal of default values in last paragraph. The first is redundant with the Arg Clinic signature, so I suggested to remove it. For the second, I would add it back if we remove the list of parameters, since it explains what the special value -1 means.
Do you want to revert all these differences?
JulienPalard commentedon Nov 25, 2016
Quoting @martin
In fact, http://docs.python.org/howto/clinic.html, bullet ".4", told me do remove it:
OK, will remove them.
Can remove it to simplify verification, but as doctests will be different as doc told me to remove the signature, so is it work removing this dot ? I don't think so.
I'll revert it too as I'll remove the documentation in the parameter list.
Will upload a patch later.
29 remaining items
JulienPalard commentedon Dec 10, 2016
Added a test to ensure compatibility of both hi=None (introduced in original Python version) and hi=-1 (Introduced by the C version).
Modified Python version to be compatible with the C-introduced hi=-1, so that the new test pass.
JulienPalard commentedon Dec 10, 2016
Maybe wait for bpo-28933.
larryhastings commentedon Dec 10, 2016
FWIW, pypy uses defaults of lo=0, hi=None.
https://bitbucket.org/pypy/pypy/src/25da7bc9719457aee57547993833d5168ba6aded/lib-python/2.7/bisect.py?at=default&fileviewer=file-view-default
JulienPalard commentedon Dec 10, 2016
Hi Larry,
In any cases it looks like supporting hi=-1 and hi=None is mandatory:
Both are currently living together, the C version takes over the Python version when available, but the Python version is older, so both -1 and None has been the "current implementation" during some time.
Raymond legitimately fear that someone, somewhere, used -1 in some code. We should not break it (even if it had never been documented).
So let's just accept both for the moment, and if deprecation of one or another should be discussed, let's do it later, in another issue. I'm _just_ trying to make AC move forward here, without breaking anything.
larryhastings commentedon Dec 10, 2016
Okay, but do it with a converter function, not by changing the default behavior for Py_ssize_t. The vast majority of uses of Py_ssize_t should not accept None.
rhettinger commentedon Dec 11, 2016
Why does this need a converter function? I there is any reason this can't use "O" with "hi" defaulting to None (matching the pure python API) and letting the C function handle both -1 and None in the implementation rather than in AC?
JulienPalard commentedon Dec 11, 2016
Hi Raymond,
I don't like having the converters in the C implementation too, that's why I'm working on bpo-28933 to clean this.
It works, yes. But I prefer to clearly split responsibilities: AC being responsible of adapting argument from PyObjects to C types, and C functions being responsible of ... doing their job.
If the idea in bpo-28933 is accepted, we'll be able to declare hi as simply as:
meaning "C function will get a Py_ssize_t, default value for C code is -1, None is documented, and None can be passed to get the C default value", that's this last point "None can be passed to get the C default value" that I introduce in bpo-28933.
With this syntax, both C converters and the python hi_parameter_converter can be dropped.
larryhastings commentedon Dec 11, 2016
We don't *have* to use a converter function, no. But consider: someone somewhere will have to convert a PyObject * into a Py_ssize_t, also accepting None. Doing it in a converter function means we can easily share the code with bisect_right, which should presumably behave the same way. It also keeps all the argument parsing logic in the Argument Clinic domain instead of having some in AC and some in the implementation.
Is there some drawback to using a converter function?
JulienPalard commentedon Dec 13, 2016
As the pattern of this converter is not widely used, I'll let it in the code of _bisect for the moment, see: http://bugs.python.org/issue28933.
serhiy-storchaka commentedon Apr 5, 2017
Now
Py_ssize_t(accept={int, NoneType})can be used instead of a local converter.rhettinger commentedon Apr 5, 2017
Wouldn't it be easily to let the OP apply AC to some other module. Right now, it isn't a very good fit and was this a tough one to start with.
rhettinger commentedon Sep 24, 2017
After more thought, I'm going to close this one because the Argument Clinic and signature objects aren't yet a good fit for this API. The bisect module isn't broken (its API has been successfully living in the wild for a very long time). I do welcome Argument Clinic patches where there is a good fit but don't think this is the place. Each of the proposed patches makes the module worst in one way or another.
rhettinger commentedon Sep 25, 2017
FWIW, the itertools module has many functions that can have the ArgumentClinic applied without any fundamental problems (pretty much all of the functions will work except for repeat(), accumulate(), and islice()).