Discussion:
[VOTE] Release PyLucene 6.5.0 (rc1) (now with Python 3 support)
Andi Vajda
2017-03-27 22:12:48 UTC
Permalink
The PyLucene 6.5.0 (rc1) release tracking today's release of
Apache Lucene 6.5.0 is ready.

A release candidate is available from:
https://dist.apache.org/repos/dist/dev/lucene/pylucene/6.5.0-rc1/

PyLucene 6.5.0 is built with JCC 3.0 included in these release artifacts.

JCC 3.0 now supports Python 3.3+ (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 6.5.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1
Ruediger Meier
2017-03-29 19:11:57 UTC
Permalink
Post by Andi Vajda
The PyLucene 6.5.0 (rc1) release tracking today's release of
Apache Lucene 6.5.0 is ready.
https://dist.apache.org/repos/dist/dev/lucene/pylucene/6.5.0-rc1/
PyLucene 6.5.0 is built with JCC 3.0 included in these release
artifacts.
JCC 3.0 now supports Python 3.3+ (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3.
Please vote to release these artifacts as PyLucene 6.5.0.
Anyone interested in this release can and should vote !
Thanks !
Andi..
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS
pps: here is my +1
+1

Thanks!

Regarding that release candidate. There are still one or two minor
issues on Linux and Windows I think. I will test again and report soon.

cu,
Rudi
Andi Vajda
2017-03-29 20:16:08 UTC
Permalink
Post by Ruediger Meier
Post by Andi Vajda
The PyLucene 6.5.0 (rc1) release tracking today's release of
Apache Lucene 6.5.0 is ready.
https://dist.apache.org/repos/dist/dev/lucene/pylucene/6.5.0-rc1/
PyLucene 6.5.0 is built with JCC 3.0 included in these release artifacts.
JCC 3.0 now supports Python 3.3+ (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3.
Please vote to release these artifacts as PyLucene 6.5.0.
Anyone interested in this release can and should vote !
Thanks !
Andi..
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS
pps: here is my +1
+1
Thanks!
Regarding that release candidate. There are still one or two minor
issues on Linux
I'm aware of the fsct that the -lpython... link line for shared mode on linux needs editing depending on the versions of python used. Are there other issues on linux ?
Post by Ruediger Meier
and Windows I think. I will test again and report soon.
I have no access to windows anymore. We (Apache committers) used to get a MSDN free subscription but that program was apparently discontinued. If you have access to windows, thank you for trying out PyLucene and JCC there !

Andi..
Post by Ruediger Meier
cu,
Rudi
Ruediger Meier
2017-03-29 20:36:25 UTC
Permalink
Post by Andi Vajda
Post by Ruediger Meier
Regarding that release candidate. There are still one or two minor
issues on Linux
I'm aware of the fsct that the -lpython... link line for shared mode
on linux needs editing depending on the versions of python used. Are
there other issues on linux ?
No other issues. I've fixed -lpython for myself like this
https://github.com/rudimeier/jcc/commit/b4a7987ebeeb96d6c71b7635160f798303715877

but you probably want do avoid the .so version earlier like you did for
OSX. You may fix it blindly as you prefer and I could test it.
Post by Andi Vajda
Post by Ruediger Meier
and Windows I think. I will test again and report soon.
I have no access to windows anymore. We (Apache committers) used to
get a MSDN free subscription but that program was apparently
discontinued. If you have access to windows, thank you for trying out
PyLucene and JCC there !
I could somehow organize a windows for testing, though it would be
painful for me. ;) I would do it shortly before final release if nobody
else does it.

So far I believe that the patch here in my last comment would do it
https://github.com/rudimeier/jcc/issues/1
(It was also part of tommykoch's port)

cu,
Rudi
Andi Vajda
2017-03-29 20:43:16 UTC
Permalink
Post by Ruediger Meier
Post by Andi Vajda
Post by Ruediger Meier
Regarding that release candidate. There are still one or two minor
issues on Linux
I'm aware of the fsct that the -lpython... link line for shared mode
on linux needs editing depending on the versions of python used. Are
there other issues on linux ?
No other issues. I've fixed -lpython for myself like this
https://github.com/rudimeier/jcc/commit/b4a7987ebeeb96d6c71b7635160f798303715877
but you probably want do avoid the .so version earlier like you did for
OSX. You may fix it blindly as you prefer and I could test it.
Post by Andi Vajda
Post by Ruediger Meier
and Windows I think. I will test again and report soon.
I have no access to windows anymore. We (Apache committers) used to
get a MSDN free subscription but that program was apparently
discontinued. If you have access to windows, thank you for trying out
PyLucene and JCC there !
I could somehow organize a windows for testing, though it would be
painful for me. ;) I would do it shortly before final release if nobody
else does it.
Don't bother. If there are no windows users willing to test this then we don't need to support windows anymore.
Post by Ruediger Meier
So far I believe that the patch here in my last comment would do it
https://github.com/rudimeier/jcc/issues/1
(It was also part of tommykoch's port)
Ok, I'll take another look...

Andi..
Post by Ruediger Meier
cu,
Rudi
Petrus Hyvönen
2017-03-29 20:52:28 UTC
Permalink
Hi,

Yes, there are windows users :)

I've ran a quick test, it builds fine on python 2.7 but I'm getting some
linker error under python 3.6 and 3.5 (didn't try lower).

The linker error states:

jcc3/sources/jcc.cpp(202): error C3688: invalid literal suffix 'PRIxMAX';
literal operator or literal operator template
'operator ""PRIxMAX' not found
jcc3/sources/jcc.cpp(202): error C2664: 'int sprintf(char *const ,const
char *const ,...)': cannot convert argument 2 fr
om 'int' to 'const char *const '
jcc3/sources/jcc.cpp(202): note: Conversion from integral type to pointer
type requires reinterpret_cast, C-style cast o
r function-style cast
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio
14.0\\VC\\BIN\\amd64\\cl.exe' failed with exit status 2

the jcc.cpp code is:
static PyObject *t_jccenv_strhash(PyObject *self, PyObject *arg)
{
static const size_t hexdig = sizeof(uintmax_t) * 2;
uintmax_t hash = (uintmax_t) PyObject_Hash(arg);
char buffer[hexdig + 1];

sprintf(buffer, "%0*"PRIxMAX, (int) hexdig, hash);
return PyUnicode_FromStringAndSize(buffer, hexdig);
}

I don't understand the PRIxMAX stuff there, what does it mean?

MANY thanks for working with the Python 3 port...

Regards
/Petrus
Post by Ruediger Meier
Post by Ruediger Meier
Post by Andi Vajda
Post by Ruediger Meier
Regarding that release candidate. There are still one or two minor
issues on Linux
I'm aware of the fsct that the -lpython... link line for shared mode
on linux needs editing depending on the versions of python used. Are
there other issues on linux ?
No other issues. I've fixed -lpython for myself like this
https://github.com/rudimeier/jcc/commit/b4a7987ebeeb96d6c71b7635160f79
8303715877
Post by Ruediger Meier
but you probably want do avoid the .so version earlier like you did for
OSX. You may fix it blindly as you prefer and I could test it.
Post by Andi Vajda
Post by Ruediger Meier
and Windows I think. I will test again and report soon.
I have no access to windows anymore. We (Apache committers) used to
get a MSDN free subscription but that program was apparently
discontinued. If you have access to windows, thank you for trying out
PyLucene and JCC there !
I could somehow organize a windows for testing, though it would be
painful for me. ;) I would do it shortly before final release if nobody
else does it.
Don't bother. If there are no windows users willing to test this then we
don't need to support windows anymore.
Post by Ruediger Meier
So far I believe that the patch here in my last comment would do it
https://github.com/rudimeier/jcc/issues/1
(It was also part of tommykoch's port)
Ok, I'll take another look...
Andi..
Post by Ruediger Meier
cu,
Rudi
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00
Petrus Hyvönen
2017-03-29 21:15:03 UTC
Permalink
Hi,

Assuming the Prixmax stuff is just some typo, if I remove that I get same
error as in Ruedigers github version - MS linker error LNK1561 - missing
starting point. This seems to be fixed by the /DLL directive.

https://msdn.microsoft.com/en-us/library/ky737ya4.aspx

and fix at
https://github.com/rudimeier/jcc/issues/1

Adding this /DLL to the linker options, I now get linker error:
"jcc3/sources/functions.cpp(303): error C3861: 'PY_SIZE': identifier not
found"

The code is:
301 int __parseArgs(PyObject *args, char *types, ...)
302 {
303: int count = PY_SIZE((PyTupleObject *) args);
304 va_list list, check;
305

The other stuff related to size i find is using the PyBytes_Size(arg)?

Again, many thanks :)

Best Regards
/Petrus
Post by Petrus Hyvönen
Hi,
Yes, there are windows users :)
I've ran a quick test, it builds fine on python 2.7 but I'm getting some
linker error under python 3.6 and 3.5 (didn't try lower).
jcc3/sources/jcc.cpp(202): error C3688: invalid literal suffix 'PRIxMAX';
literal operator or literal operator template
'operator ""PRIxMAX' not found
jcc3/sources/jcc.cpp(202): error C2664: 'int sprintf(char *const ,const
char *const ,...)': cannot convert argument 2 fr
om 'int' to 'const char *const '
jcc3/sources/jcc.cpp(202): note: Conversion from integral type to pointer
type requires reinterpret_cast, C-style cast o
r function-style cast
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio
14.0\\VC\\BIN\\amd64\\cl.exe' failed with exit status 2
static PyObject *t_jccenv_strhash(PyObject *self, PyObject *arg)
{
static const size_t hexdig = sizeof(uintmax_t) * 2;
uintmax_t hash = (uintmax_t) PyObject_Hash(arg);
char buffer[hexdig + 1];
sprintf(buffer, "%0*"PRIxMAX, (int) hexdig, hash);
return PyUnicode_FromStringAndSize(buffer, hexdig);
}
I don't understand the PRIxMAX stuff there, what does it mean?
MANY thanks for working with the Python 3 port...
Regards
/Petrus
Post by Ruediger Meier
Post by Ruediger Meier
Post by Andi Vajda
Post by Ruediger Meier
Regarding that release candidate. There are still one or two minor
issues on Linux
I'm aware of the fsct that the -lpython... link line for shared mode
on linux needs editing depending on the versions of python used. Are
there other issues on linux ?
No other issues. I've fixed -lpython for myself like this
https://github.com/rudimeier/jcc/commit/b4a7987ebeeb96d6c71b
7635160f798303715877
Post by Ruediger Meier
but you probably want do avoid the .so version earlier like you did for
OSX. You may fix it blindly as you prefer and I could test it.
Post by Andi Vajda
Post by Ruediger Meier
and Windows I think. I will test again and report soon.
I have no access to windows anymore. We (Apache committers) used to
get a MSDN free subscription but that program was apparently
discontinued. If you have access to windows, thank you for trying out
PyLucene and JCC there !
I could somehow organize a windows for testing, though it would be
painful for me. ;) I would do it shortly before final release if nobody
else does it.
Don't bother. If there are no windows users willing to test this then we
don't need to support windows anymore.
Post by Ruediger Meier
So far I believe that the patch here in my last comment would do it
https://github.com/rudimeier/jcc/issues/1
(It was also part of tommykoch's port)
Ok, I'll take another look...
Andi..
Post by Ruediger Meier
cu,
Rudi
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00 <073-803%2019%2000>
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00
Ruediger Meier
2017-03-29 21:26:37 UTC
Permalink
Post by Petrus Hyvönen
Hi,
Assuming the Prixmax stuff is just some typo, if I remove that I get
same error as in Ruedigers github version - MS linker error LNK1561 -
missing starting point. This seems to be fixed by the /DLL
directive.
https://msdn.microsoft.com/en-us/library/ky737ya4.aspx
and fix at
https://github.com/rudimeier/jcc/issues/1
"jcc3/sources/functions.cpp(303): error C3861: 'PY_SIZE': identifier
not found"
301 int __parseArgs(PyObject *args, char *types, ...)
302 {
303: int count = PY_SIZE((PyTupleObject *) args);
304 va_list list, check;
305
Probably just a typo, try "Py_SIZE" (case sensitive "y").

cu,
Rudi
Petrus Hyvönen
2017-03-29 21:53:25 UTC
Permalink
Hi,

With the /DLL, sprintf(buffer, "%0*%jx", (int) hexdig, hash); and Py_SIZE
it compiles under windows (Windows 7, 64 bit)

I haven't set up for building pylucene but has another library that I build.

For that I get a udf-8 error on:

File
"C:\Users\phy\AppData\Local\Continuum\Anaconda3-430\conda-bld\orekit_1490824040916\_b_env\lib\site-packages\jcc\c
pp.py", line 898, in header
env.strhash(signature(constructor)))
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xac in position 9:
invalid start byte

I will do more tests later, and see if I can test with building pylucene.

Regards
/Petrus
Post by Ruediger Meier
Post by Petrus Hyvönen
Hi,
Assuming the Prixmax stuff is just some typo, if I remove that I get
same error as in Ruedigers github version - MS linker error LNK1561 -
missing starting point. This seems to be fixed by the /DLL
directive.
https://msdn.microsoft.com/en-us/library/ky737ya4.aspx
and fix at
https://github.com/rudimeier/jcc/issues/1
"jcc3/sources/functions.cpp(303): error C3861: 'PY_SIZE': identifier
not found"
301 int __parseArgs(PyObject *args, char *types, ...)
302 {
303: int count = PY_SIZE((PyTupleObject *) args);
304 va_list list, check;
305
Probably just a typo, try "Py_SIZE" (case sensitive "y").
cu,
Rudi
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00
Andi Vajda
2017-03-29 21:59:10 UTC
Permalink
Post by Petrus Hyvönen
Hi,
With the /DLL, sprintf(buffer, "%0*%jx", (int) hexdig, hash); and Py_SIZE
it compiles under windows (Windows 7, 64 bit)
I haven't set up for building pylucene but has another library that I build.
File
"C:\Users\phy\AppData\Local\Continuum\Anaconda3-430\conda-bld\orekit_1490824040916\_b_env\lib\site-packages\jcc\c
pp.py", line 898, in header
env.strhash(signature(constructor)))
invalid start byte
That means that the "%0*%jx" change for _MSC_VER also needs a change for the
hexdig sizeof computation. There is a mismatch and garbage is left in the
buffer array.

Andi..
Post by Petrus Hyvönen
I will do more tests later, and see if I can test with building pylucene.
Regards
/Petrus
Post by Ruediger Meier
Post by Petrus Hyvönen
Hi,
Assuming the Prixmax stuff is just some typo, if I remove that I get
same error as in Ruedigers github version - MS linker error LNK1561 -
missing starting point. This seems to be fixed by the /DLL
directive.
https://msdn.microsoft.com/en-us/library/ky737ya4.aspx
and fix at
https://github.com/rudimeier/jcc/issues/1
"jcc3/sources/functions.cpp(303): error C3861: 'PY_SIZE': identifier
not found"
301 int __parseArgs(PyObject *args, char *types, ...)
302 {
303: int count = PY_SIZE((PyTupleObject *) args);
304 va_list list, check;
305
Probably just a typo, try "Py_SIZE" (case sensitive "y").
cu,
Rudi
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00
Ruediger Meier
2017-03-29 22:48:26 UTC
Permalink
Post by Andi Vajda
Post by Petrus Hyvönen
Hi,
With the /DLL, sprintf(buffer, "%0*%jx", (int) hexdig, hash); and
Py_SIZE it compiles under windows (Windows 7, 64 bit)
I haven't set up for building pylucene but has another library that I build.
File
"C:\Users\phy\AppData\Local\Continuum\Anaconda3-430\conda-bld\oreki
t_1490824040916\_b_env\lib\site-packages\jcc\c pp.py", line 898, in
header
env.strhash(signature(constructor)))
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xac in
position 9: invalid start byte
That means that the "%0*%jx" change for _MSC_VER also needs a change
for the hexdig sizeof computation. There is a mismatch and garbage is
left in the buffer array.
Hm, actually "%jx" and PRIxMAX should be equivalent modifiers to print
uintmax_t (c99). According to the MSC documentation I thought that the
macro is even more safe to use.

The problem is that we do C++ here which does not need to support c99 at
all. Though MSC might be the only C++ compiler which ignores c99...

To fix it I would suggest to use non-c99 types here. Lets use "unsigned
long long" and "%llx". This may look a bit less rock-solid but good
enough for any existing systems:

static PyObject *t_jccenv_strhash(PyObject *self, PyObject *arg)
{
unsigned long long hash = (unsigned long long) PyObject_Hash(arg);
static const size_t hexdig = sizeof(hash) * 2;
char buffer[hexdig + 1];

sprintf(buffer, "%0*llx", (int) hexdig, hash);
return PyUnicode_FromStringAndSize(buffer, hexdig);
}

BTW this function should be also copied to the py2 directory where we
still use int allthough PyObject_Hash returns already long on python
Post by Andi Vajda
2.x.
cu,
Rudi
Andi Vajda
2017-03-29 23:04:15 UTC
Permalink
Post by Ruediger Meier
Post by Andi Vajda
Post by Petrus Hyvönen
Hi,
With the /DLL, sprintf(buffer, "%0*%jx", (int) hexdig, hash); and
Py_SIZE it compiles under windows (Windows 7, 64 bit)
I haven't set up for building pylucene but has another library that I build.
File
"C:\Users\phy\AppData\Local\Continuum\Anaconda3-430\conda-bld\oreki
t_1490824040916\_b_env\lib\site-packages\jcc\c pp.py", line 898, in
header
env.strhash(signature(constructor)))
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xac in
position 9: invalid start byte
That means that the "%0*%jx" change for _MSC_VER also needs a change
for the hexdig sizeof computation. There is a mismatch and garbage is
left in the buffer array.
Hm, actually "%jx" and PRIxMAX should be equivalent modifiers to print
uintmax_t (c99). According to the MSC documentation I thought that the
macro is even more safe to use.
The problem is that we do C++ here which does not need to support c99 at
all. Though MSC might be the only C++ compiler which ignores c99...
To fix it I would suggest to use non-c99 types here. Lets use "unsigned
long long" and "%llx". This may look a bit less rock-solid but good
I'm happy to hide this _MSC_VER hackery in conditional code.

Andi..
Post by Ruediger Meier
static PyObject *t_jccenv_strhash(PyObject *self, PyObject *arg)
{
unsigned long long hash = (unsigned long long) PyObject_Hash(arg);
static const size_t hexdig = sizeof(hash) * 2;
char buffer[hexdig + 1];
sprintf(buffer, "%0*llx", (int) hexdig, hash);
return PyUnicode_FromStringAndSize(buffer, hexdig);
}
BTW this function should be also copied to the py2 directory where we
still use int allthough PyObject_Hash returns already long on python
Post by Andi Vajda
2.x.
cu,
Rudi
Petrus Hyvönen
2017-03-30 07:10:02 UTC
Permalink
Hi,

With this version of of t_jccenv_strhash I can build both JCC and wrap the
library I'm using!

Regards
/Petrus
Post by Petrus Hyvönen
static PyObject *t_jccenv_strhash(PyObject *self, PyObject *arg)
{
unsigned long long hash = (unsigned long long) PyObject_Hash(arg);
static const size_t hexdig = sizeof(hash) * 2;
char buffer[hexdig + 1];
sprintf(buffer, "%0*llx", (int) hexdig, hash);
return PyUnicode_FromStringAndSize(buffer, hexdig);
}
BTW this function should be also copied to the py2 directory where we
still use int allthough PyObject_Hash returns already long on python
2.x.
cu,
Rudi
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00
Petrus Hyvönen
2017-03-30 15:36:05 UTC
Permalink
Hi,

I was trying the python 2.7 build and I think the line 23 in
jcc2/__init__.py should be:

from jcc.config import SHARED

(instead of from jcc2.config import..)

Regards
/Petrus
Post by Petrus Hyvönen
Hi,
With this version of of t_jccenv_strhash I can build both JCC and wrap the
library I'm using!
Regards
/Petrus
Post by Petrus Hyvönen
static PyObject *t_jccenv_strhash(PyObject *self, PyObject *arg)
{
unsigned long long hash = (unsigned long long) PyObject_Hash(arg);
static const size_t hexdig = sizeof(hash) * 2;
char buffer[hexdig + 1];
sprintf(buffer, "%0*llx", (int) hexdig, hash);
return PyUnicode_FromStringAndSize(buffer, hexdig);
}
BTW this function should be also copied to the py2 directory where we
still use int allthough PyObject_Hash returns already long on python
2.x.
cu,
Rudi
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00 <073-803%2019%2000>
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00
Petrus Hyvönen
2017-03-30 16:11:10 UTC
Permalink
Hi,

My current diff to the svn is below (as in the chain of mails). Now i get
it to wrap my library in both 2.7, 3.5 and 3.6.

/Regards


Index: jcc2/__init__.py
===================================================================
--- jcc2/__init__.py (revision 1789413)
+++ jcc2/__init__.py (working copy)
@@ -20,7 +20,7 @@
from windows import add_jvm_dll_directory_to_path
add_jvm_dll_directory_to_path()

- from jcc2.config import SHARED
+ from jcc.config import SHARED
if SHARED:
path = os.environ['Path'].split(os.pathsep)
eggpath =
os.path.abspath(os.path.dirname(os.path.dirname(__file__)))
Index: jcc3/sources/functions.cpp
===================================================================
--- jcc3/sources/functions.cpp (revision 1789413)
+++ jcc3/sources/functions.cpp (working copy)
@@ -300,7 +300,7 @@
#if defined(_MSC_VER) || defined(__SUNPRO_CC)
int __parseArgs(PyObject *args, char *types, ...)
{
- int count = PY_SIZE((PyTupleObject *) args);
+ int count = Py_SIZE((PyTupleObject *) args); //WAS PY_SIZE
va_list list, check;

va_start(list, types);
Index: jcc3/sources/jcc.cpp
===================================================================
--- jcc3/sources/jcc.cpp (revision 1789413)
+++ jcc3/sources/jcc.cpp (working copy)
@@ -195,11 +195,11 @@

static PyObject *t_jccenv_strhash(PyObject *self, PyObject *arg)
{
- static const size_t hexdig = sizeof(uintmax_t) * 2;
- uintmax_t hash = (uintmax_t) PyObject_Hash(arg);
+ unsigned long long hash = (unsigned long long) PyObject_Hash(arg);
+ static const size_t hexdig = sizeof(hash) * 2;
char buffer[hexdig + 1];

- sprintf(buffer, "%0*"PRIxMAX, (int) hexdig, hash);
+ sprintf(buffer, "%0*llx", (int) hexdig, hash);
return PyUnicode_FromStringAndSize(buffer, hexdig);
}

Index: setup.py
===================================================================
--- setup.py (revision 1789413)
+++ setup.py (working copy)
@@ -158,7 +158,7 @@
'sunos5': ['-L%(sunos5)s/jre/lib/i386' %(JDK), '-ljava',
'-L%(sunos5)s/jre/lib/i386/client' %(JDK), '-ljvm',
'-R%(sunos5)s/jre/lib/i386:%(sunos5)s/jre/lib/i386/client'
%(JDK)],
- 'win32': ['/LIBPATH:%(win32)s/lib' %(JDK), 'Ws2_32.lib', 'jvm.lib'],
+ 'win32': ['/LIBPATH:%(win32)s/lib' %(JDK), 'Ws2_32.lib',
'jvm.lib','/DLL'],
'mingw32': ['-L%(mingw32)s/lib' %(JDK), '-ljvm'],
'freebsd7': ['-L%(freebsd7)s/jre/lib/i386' %(JDK), '-ljava',
'-lverify',
'-L%(freebsd7)s/jre/lib/i386/client' %(JDK), '-ljvm',
Post by Petrus Hyvönen
Hi,
I was trying the python 2.7 build and I think the line 23 in
from jcc.config import SHARED
(instead of from jcc2.config import..)
Regards
/Petrus
Post by Petrus Hyvönen
Hi,
With this version of of t_jccenv_strhash I can build both JCC and wrap
the library I'm using!
Regards
/Petrus
Post by Petrus Hyvönen
static PyObject *t_jccenv_strhash(PyObject *self, PyObject *arg)
{
unsigned long long hash = (unsigned long long) PyObject_Hash(arg);
static const size_t hexdig = sizeof(hash) * 2;
char buffer[hexdig + 1];
sprintf(buffer, "%0*llx", (int) hexdig, hash);
return PyUnicode_FromStringAndSize(buffer, hexdig);
}
BTW this function should be also copied to the py2 directory where we
still use int allthough PyObject_Hash returns already long on python
2.x.
cu,
Rudi
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00 <073-803%2019%2000>
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00 <073-803%2019%2000>
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00
Andi Vajda
2017-03-30 19:05:41 UTC
Permalink
Post by Petrus Hyvönen
Hi,
My current diff to the svn is below (as in the chain of mails). Now i get
it to wrap my library in both 2.7, 3.5 and 3.6.
I believe, I've now applied all these diffs (or equivalents).
Thank you Petrus for testing on Windows, I'm going to release rc2 artifacts
and call for a new vote.

Andi..
Post by Petrus Hyvönen
/Regards
Index: jcc2/__init__.py
===================================================================
--- jcc2/__init__.py (revision 1789413)
+++ jcc2/__init__.py (working copy)
@@ -20,7 +20,7 @@
from windows import add_jvm_dll_directory_to_path
add_jvm_dll_directory_to_path()
- from jcc2.config import SHARED
+ from jcc.config import SHARED
path = os.environ['Path'].split(os.pathsep)
eggpath =
os.path.abspath(os.path.dirname(os.path.dirname(__file__)))
Index: jcc3/sources/functions.cpp
===================================================================
--- jcc3/sources/functions.cpp (revision 1789413)
+++ jcc3/sources/functions.cpp (working copy)
@@ -300,7 +300,7 @@
#if defined(_MSC_VER) || defined(__SUNPRO_CC)
int __parseArgs(PyObject *args, char *types, ...)
{
- int count = PY_SIZE((PyTupleObject *) args);
+ int count = Py_SIZE((PyTupleObject *) args); //WAS PY_SIZE
va_list list, check;
va_start(list, types);
Index: jcc3/sources/jcc.cpp
===================================================================
--- jcc3/sources/jcc.cpp (revision 1789413)
+++ jcc3/sources/jcc.cpp (working copy)
@@ -195,11 +195,11 @@
static PyObject *t_jccenv_strhash(PyObject *self, PyObject *arg)
{
- static const size_t hexdig = sizeof(uintmax_t) * 2;
- uintmax_t hash = (uintmax_t) PyObject_Hash(arg);
+ unsigned long long hash = (unsigned long long) PyObject_Hash(arg);
+ static const size_t hexdig = sizeof(hash) * 2;
char buffer[hexdig + 1];
- sprintf(buffer, "%0*"PRIxMAX, (int) hexdig, hash);
+ sprintf(buffer, "%0*llx", (int) hexdig, hash);
return PyUnicode_FromStringAndSize(buffer, hexdig);
}
Index: setup.py
===================================================================
--- setup.py (revision 1789413)
+++ setup.py (working copy)
@@ -158,7 +158,7 @@
'sunos5': ['-L%(sunos5)s/jre/lib/i386' %(JDK), '-ljava',
'-L%(sunos5)s/jre/lib/i386/client' %(JDK), '-ljvm',
'-R%(sunos5)s/jre/lib/i386:%(sunos5)s/jre/lib/i386/client'
%(JDK)],
- 'win32': ['/LIBPATH:%(win32)s/lib' %(JDK), 'Ws2_32.lib', 'jvm.lib'],
+ 'win32': ['/LIBPATH:%(win32)s/lib' %(JDK), 'Ws2_32.lib',
'jvm.lib','/DLL'],
'mingw32': ['-L%(mingw32)s/lib' %(JDK), '-ljvm'],
'freebsd7': ['-L%(freebsd7)s/jre/lib/i386' %(JDK), '-ljava',
'-lverify',
'-L%(freebsd7)s/jre/lib/i386/client' %(JDK), '-ljvm',
Post by Petrus Hyvönen
Hi,
I was trying the python 2.7 build and I think the line 23 in
from jcc.config import SHARED
(instead of from jcc2.config import..)
Regards
/Petrus
Post by Petrus Hyvönen
Hi,
With this version of of t_jccenv_strhash I can build both JCC and wrap
the library I'm using!
Regards
/Petrus
Post by Petrus Hyvönen
static PyObject *t_jccenv_strhash(PyObject *self, PyObject *arg)
{
unsigned long long hash = (unsigned long long) PyObject_Hash(arg);
static const size_t hexdig = sizeof(hash) * 2;
char buffer[hexdig + 1];
sprintf(buffer, "%0*llx", (int) hexdig, hash);
return PyUnicode_FromStringAndSize(buffer, hexdig);
}
BTW this function should be also copied to the py2 directory where we
still use int allthough PyObject_Hash returns already long on python
2.x.
cu,
Rudi
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00 <073-803%2019%2000>
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00 <073-803%2019%2000>
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00
Rüdiger Meier
2017-03-30 19:18:20 UTC
Permalink
Post by Petrus Hyvönen
Hi,
My current diff to the svn is below (as in the chain of mails). Now
i get it to wrap my library in both 2.7, 3.5 and 3.6.
I believe, I've now applied all these diffs (or equivalents). Thank
you Petrus for testing on Windows, I'm going to release rc2 artifacts
and call for a new vote.
Just noticed another minor thing needed for python 3.7 support.

jcc3/sources/jcc.cpp, line 485:

- char *option = PyUnicode_AsUTF8(arg);
+ const char *option = PyUnicode_AsUTF8(arg);

cu,
Rudi
Andi Vajda
2017-03-30 19:25:05 UTC
Permalink
Post by Rüdiger Meier
Post by Petrus Hyvönen
Hi,
My current diff to the svn is below (as in the chain of mails). Now
i get it to wrap my library in both 2.7, 3.5 and 3.6.
I believe, I've now applied all these diffs (or equivalents). Thank
you Petrus for testing on Windows, I'm going to release rc2 artifacts
and call for a new vote.
Just noticed another minor thing needed for python 3.7 support.
There is no Python 3.7 yet, is there ?
Anyhow, I just cut rc2. Next release.

Andi..
Post by Rüdiger Meier
- char *option = PyUnicode_AsUTF8(arg);
+ const char *option = PyUnicode_AsUTF8(arg);
cu,
Rudi
Andi Vajda
2017-03-29 21:59:53 UTC
Permalink
Post by Petrus Hyvönen
Hi,
Assuming the Prixmax stuff is just some typo, if I remove that I get same
error as in Ruedigers github version - MS linker error LNK1561 - missing
starting point. This seems to be fixed by the /DLL directive.
https://msdn.microsoft.com/en-us/library/ky737ya4.aspx
and fix at
https://github.com/rudimeier/jcc/issues/1
"jcc3/sources/functions.cpp(303): error C3861: 'PY_SIZE': identifier not
found"
301 int __parseArgs(PyObject *args, char *types, ...)
302 {
303: int count = PY_SIZE((PyTupleObject *) args);
Ooop, that should be Py_SIZE. Fixed now.

Andi..
Post by Petrus Hyvönen
304 va_list list, check;
305
The other stuff related to size i find is using the PyBytes_Size(arg)?
Again, many thanks :)
Best Regards
/Petrus
Post by Petrus Hyvönen
Hi,
Yes, there are windows users :)
I've ran a quick test, it builds fine on python 2.7 but I'm getting some
linker error under python 3.6 and 3.5 (didn't try lower).
jcc3/sources/jcc.cpp(202): error C3688: invalid literal suffix 'PRIxMAX';
literal operator or literal operator template
'operator ""PRIxMAX' not found
jcc3/sources/jcc.cpp(202): error C2664: 'int sprintf(char *const ,const
char *const ,...)': cannot convert argument 2 fr
om 'int' to 'const char *const '
jcc3/sources/jcc.cpp(202): note: Conversion from integral type to pointer
type requires reinterpret_cast, C-style cast o
r function-style cast
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio
14.0\\VC\\BIN\\amd64\\cl.exe' failed with exit status 2
static PyObject *t_jccenv_strhash(PyObject *self, PyObject *arg)
{
static const size_t hexdig = sizeof(uintmax_t) * 2;
uintmax_t hash = (uintmax_t) PyObject_Hash(arg);
char buffer[hexdig + 1];
sprintf(buffer, "%0*"PRIxMAX, (int) hexdig, hash);
return PyUnicode_FromStringAndSize(buffer, hexdig);
}
I don't understand the PRIxMAX stuff there, what does it mean?
MANY thanks for working with the Python 3 port...
Regards
/Petrus
Post by Ruediger Meier
Post by Ruediger Meier
Post by Andi Vajda
Post by Ruediger Meier
Regarding that release candidate. There are still one or two minor
issues on Linux
I'm aware of the fsct that the -lpython... link line for shared mode
on linux needs editing depending on the versions of python used. Are
there other issues on linux ?
No other issues. I've fixed -lpython for myself like this
https://github.com/rudimeier/jcc/commit/b4a7987ebeeb96d6c71b
7635160f798303715877
Post by Ruediger Meier
but you probably want do avoid the .so version earlier like you did for
OSX. You may fix it blindly as you prefer and I could test it.
Post by Andi Vajda
Post by Ruediger Meier
and Windows I think. I will test again and report soon.
I have no access to windows anymore. We (Apache committers) used to
get a MSDN free subscription but that program was apparently
discontinued. If you have access to windows, thank you for trying out
PyLucene and JCC there !
I could somehow organize a windows for testing, though it would be
painful for me. ;) I would do it shortly before final release if nobody
else does it.
Don't bother. If there are no windows users willing to test this then we
don't need to support windows anymore.
Post by Ruediger Meier
So far I believe that the patch here in my last comment would do it
https://github.com/rudimeier/jcc/issues/1
(It was also part of tommykoch's port)
Ok, I'll take another look...
Andi..
Post by Ruediger Meier
cu,
Rudi
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00 <073-803%2019%2000>
--
_____________________________________________
Petrus Hyvönen, Uppsala, Sweden
Mobile Phone/SMS:+46 73 803 19 00
Ruediger Meier
2017-03-29 21:18:13 UTC
Permalink
Post by Petrus Hyvönen
Hi,
Yes, there are windows users :)
I've ran a quick test, it builds fine on python 2.7 but I'm getting
some linker error under python 3.6 and 3.5 (didn't try lower).
jcc3/sources/jcc.cpp(202): error C3688: invalid literal suffix
'PRIxMAX'; literal operator or literal operator template
'operator ""PRIxMAX' not found
jcc3/sources/jcc.cpp(202): error C2664: 'int sprintf(char *const
,const char *const ,...)': cannot convert argument 2 fr
om 'int' to 'const char *const '
jcc3/sources/jcc.cpp(202): note: Conversion from integral type to
pointer type requires reinterpret_cast, C-style cast o
r function-style cast
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio
14.0\\VC\\BIN\\amd64\\cl.exe' failed with exit status 2
static PyObject *t_jccenv_strhash(PyObject *self, PyObject *arg)
{
static const size_t hexdig = sizeof(uintmax_t) * 2;
uintmax_t hash = (uintmax_t) PyObject_Hash(arg);
char buffer[hexdig + 1];
sprintf(buffer, "%0*"PRIxMAX, (int) hexdig, hash);
return PyUnicode_FromStringAndSize(buffer, hexdig);
}
I don't understand the PRIxMAX stuff there, what does it mean?
Could you try to change
sprintf(buffer, "%0*"PRIxMAX, (int) hexdig, hash);
to
sprintf(buffer, "%0*%jx", (int) hexdig, hash);

or if it still not works
sprintf(buffer, "%0*%llx", (int) hexdig, (unsigned long long)hash);

cu,
Rudi
Loading...