When you say that virtualenv and pip serve the same desire as me, you would have to give me more details, because I don’t understand how that’s possible, the goals being so different. What I demand from a deployment tool is a very controlled way to specify each step of the installation, and something which integrates well with the target platform. Ideally, it should be resilient against failure, with rollback mode, and should have a query system to deal with different kind of dependencies. How virtualenv/pip/buildout can help me to produce something I can redistribute to many people ? I myself use Virtualenv for developing quite a bit, I think that’s a useful tool in that context. I But for deployment, I don’t see it ? I may just use the tool backward. Whenever I saw a description on how people use it, it was antithetic to how I think about deployment, though (put everything in one self-contained directory – which is why I refered to the rail post).

But virtualenv, pip, and buildout are all based on the same basic desires as you: get something running reliably on a variety of systems, and make it consistent over different machines and over time. And these tools aren’t just about hacking something together that works for one person, they are very much about serving *lots* of people… because while we’re web developers, we’re also developing reusable open source libraries, and it’s probably that second role that drives the direction more than the first.

I read the Rails post, and it’s *not* how we’re going about things. I do prefer isolation and independence at the application level. I do give pushback to people who “require” a particular installation scheme for principled rather than practical reasons. But these are robust and repeatable systems that don’t punt on the issues he’s talking about.

The program files (x86) issue is very weird: where did you install python ?

As to why not building with VS 2008 – as long as the issue is not fixed for both numpy and scipy, there is no point in publishing new builds, because later ones may be incompatible if something changes again in the toolchain. Once the choice on which toolchain will be used for 64 bits binaries will be made, new binaries will be published. Numpy is easy enough that if you need numpy, you may just do it by yourself.

I recently came across a small post about the ruby gems situation (http://www.madstop.com/ruby/ruby_has_a_distribution_problem.html) which summarizes the problem quite well (you can replace rails community with setuptools/virtualenv community): “This is basically anathema to how I think about management, yet it’s the standard, recommended practice in the Rails community, because it makes it easy to “guarantee” behaviour in a given environment. Of course, your guarantee is only good if no one ever tries to run the software anywhere except an exact duplicate of where you run it.”

This sentence nails it IMHO.

I’d like to see package trees made more *visible*:
- list all directories and single files from “install X”
- look at a tree of required packages, their requires … before installing: a tiny database as dbs go, displaying subtrees with “X>= 1.2″ easy

$ python
Python 2.6.2 (r262:71605, Apr 14 2009, 22:46:50) [MSC v.1500 64 bit (AMD64)] on win32

also, if what you say works, why isn’t this distributed, rather than the numpy-1.3.0.win-amd64-py2.6.msi file which is broken. for one thing, it puts the modules into the Program Files (x86) folder instead of the normal Program Files. when those files/folders are moved over by hand, numpy crashes while loading lapack_lite.

since i just got a new Vista 64-bit machine and use Windows (more than i would like to), i am game to do a proper build, if the instructions make sense and are consistent with other web lore. so if your scheme works, I’d like to try it and then have a scheme for getting it packaged properly for distribution, so others don’t have to go thru this.

my team are currently working on cross compilation to create an executable program of minisip from linux to windows using i586-mingw32msvc-gcc,what the problem here is that the created .exe is not working with wine. here is the sample script that we using to build the .exe

#! /usr/bin/env bash
#
#This scripts is a modified version of the buildall.script.
#Thi script compiles and install all the minisip libraries and GTK GUI
# for a win32 system.
#Modify the parameters (basically the prefix_*) to adapt to your cross-compile environment.
#It is not really good if you are developing or debugging, as everytime you run the script it
# recompiles ALL the source code.
#This options are added for the w32 compilation … modify if needed, specially the cross-compile folder
#This is the CROSS_COMPILE_FOLDER
prefix_cross=”/usr/local/cross”
prefix_cross_include=”$prefix_cross/include”
prefix_cross_lib=”$prefix_cross/lib”
host_type_name=”i586-mingw32msvc”
host_option=”–host=$host_type_name”
build_option=”–build=i686-pc-linux-gnu”
strip_bin=”i586-mingw32msvc-strip”
#strip_generated_files=”yes”
# Simple script to build minisip.
#
# This script needs to be executed at the root of the repository trunk. It
# will build minisip and with all the associated libraries. The build will be
# done “in place”, that is, no files will be installed on your system.
#
# Instructions:
# Use buildall.sh script the first time you get the sources of minisip. It will
# create all files needed and compile all the sources.
# Then, if you modify a file in the sources and do not want to recompile all
# MiniSIP again, use the “buildonly.sh” script
# Note that you con modify the configuration of minisip by editing the
# configure_params variable in this script. Some useful options are commented
# out by default, but feel free to use them.
SUBDIRS=”${SUBDIRS} libmutil”
SUBDIRS=”${SUBDIRS} libmnetutil”
SUBDIRS=”${SUBDIRS} libmcrypto”
SUBDIRS=”${SUBDIRS} libmikey”
SUBDIRS=”${SUBDIRS} libmstun”
SUBDIRS=”${SUBDIRS} libmsip”
SUBDIRS=”${SUBDIRS} libzrtpcpp”
SUBDIRS=”${SUBDIRS} libminisip”
SUBDIRS=”${SUBDIRS} minisip”
#If you are debugging, you probably want to build with “g++ -g”, to build
#with all debug symbols, useful for coredumps with gdb.
#Comment this line if you don’t want to build with debug code
compiler_debug=”-ggdb”
#Also useful, you may want to call make with some options … supply them
#here.
#For example, -k forces make to keep compiling even there are errors in the
# sources. I like this one.
make_options=”-k”
#Possible configure commands are shown in: ./configure –help
# check each folder for more details.
#This are common params, usable in all folders
base_configure_params=”"
base_configure_params=”$base_configure_params –enable-debug”
base_configure_params=”$base_configure_params –disable-static”
#base_configure_params=”$base_configure_params –disable-shared”
#set special options for libmutil
libmutil_configure_params=”"
libmutil_configure_params=”$libmutil_configure_params –enable-memdebug”
#set special options for libminisip
# do a ./configure –help to see ALL available options … here
# show just a sample
libminisip_configure_params=”"
libminisip_configure_params=”$libminisip_configure_params –enable-dsound”
libminisip_configure_params=”$libminisip_configure_params –disable-alsa”
libminisip_configure_params=”$libminisip_configure_params –disable-gconf”
#–enable-autocall enables automatic calling for debug purposes (default disabled)
#–enable-ipaq enables various fixes for the iPAQ (default disabled)
#–enable-ipsec-enable enables ipsec features (default disabled)
#–enable-aec enables push-2-talk features (default enabled)
#–enable-video enables video features (default disabled)
#–enable-buzzer enables IPAQ buzzer (default disabled)
#–enable-dsound enables DirectSound sound support (default disabled)
#–enable-portaudio enable PortAudio V19 support (default auto).
#–enable-gconf enables support for GConf (default enabled).
#–enable-sdl enables SDL video output if the required library is found
#–enable-video enables video features (default disabled)
#set special options for minisip
# do a ./configure –help to see ALL available options … here
# show just a sample
minisip_configure_params=”"
minisip_configure_params=”$minisip_configure_params –enable-gtk”
minisip_configure_params=”$minisip_configure_params –disable-shared”
#minisip_configure_params=”$minisip_configure_params –enable-textui”
#Here you could have per gui-specific parameters …
for subdir in ${SUBDIRS}
do
echo “+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++”
echo “Building ${subdir} … ”
echo “+++++++++++++++++++++++++++++++++++++”
cd ${subdir}
./bootstrap
configure_params=”$base_configure_params”
if [ ${subdir} = "libmutil" ]; then
LOC_MUTIL_LIBS=-L$PWD/.libs
#${subdir}.$LIBS_EXTENSION
LOC_MUTIL_CFLAGS=-I$PWD/include
echo libmutil can use special params
configure_params=”$configure_params $libmutil_configure_params”
MUTIL_LIBS=”$LOC_MUTIL_LIBS -lmutil” \
MUTIL_CFLAGS=$LOC_MUTIL_CFLAGS \
CPPFLAGS=”-I$prefix_cross_include ” \
CXXFLAGS=”-Wall $compiler_debug” \
LDFLAGS=”-L$prefix_cross_lib” \
./configure $host_option $build_option prefix=$prefix_cross $configure_params
fi
if [ ${subdir} = "libmnetutil" ]; then
LOC_MNETUTIL_LIBS=-L$PWD/.libs
LOC_MNETUTIL_CFLAGS=-I$PWD/include
MUTIL_LIBS=”$LOC_MUTIL_LIBS -lmutil” \
MUTIL_CFLAGS=$LOC_MUTIL_CFLAGS \
CPPFLAGS=”-I$prefix_cross_include ” \
CXXFLAGS=”-Wall $compiler_debug” \
LDFLAGS=”-L$prefix_cross_lib” \
./configure $host_option $build_option prefix=$prefix_cross $configure_params
fi
if [ ${subdir} = "libmcrypto" ]; then
LOC_MCRYPTO_LIBS=-L$PWD/.libs
LOC_MCRYPTO_CFLAGS=-I$PWD/include
MUTIL_LIBS=”$LOC_MUTIL_LIBS -lmutil” \
MUTIL_CFLAGS=$LOC_MUTIL_CFLAGS \
CPPFLAGS=”-I$prefix_cross_include ” \
CXXFLAGS=”-Wall $compiler_debug” \
LDFLAGS=”-L$prefix_cross_lib” \
./configure $host_option $build_option prefix=$prefix_cross $configure_params
fi
if [ ${subdir} = "libmikey" ]; then
LOC_MIKEY_LIBS=-L$PWD/.libs
LOC_MIKEY_CFLAGS=-I$PWD/include
MUTIL_LIBS=”$LOC_MUTIL_LIBS -lmutil” \
MUTIL_CFLAGS=$LOC_MUTIL_CFLAGS \
MNETUTIL_LIBS=”$LOC_MNETUTIL_LIBS -lmnetutil” \
MNETUTIL_CFLAGS=$LOC_MNETUTIL_CFLAGS \
CPPFLAGS=”-I$prefix_cross_include ” \
CXXFLAGS=”-Wall $compiler_debug” \
LDFLAGS=”-L$prefix_cross_lib” \
./configure $host_option $build_option prefix=$prefix_cross $configure_params
fi
if [ ${subdir} = "libmstun" ]; then
LOC_MSTUN_LIBS=-L$PWD/.libs
LOC_MSTUN_CFLAGS=-I$PWD/include
MUTIL_LIBS=”$LOC_MUTIL_LIBS -lmutil” \
MUTIL_CFLAGS=$LOC_MUTIL_CFLAGS \
MNETUTIL_LIBS=”$LOC_MNETUTIL_LIBS -lmnetutil” \
MNETUTIL_CFLAGS=$LOC_MNETUTIL_CFLAGS \
CPPFLAGS=”-I$prefix_cross_include ” \
CXXFLAGS=”-Wall $compiler_debug” \
LDFLAGS=”-L$prefix_cross_lib” \
./configure $host_option $build_option prefix=$prefix_cross $configure_params
fi
if [ ${subdir} = "libmsip" ]; then
LOC_MSIP_LIBS=-L$PWD/.libs
LOC_MSIP_CFLAGS=-I$PWD/include
MUTIL_LIBS=”$LOC_MUTIL_LIBS -lmutil” \
MUTIL_CFLAGS=$LOC_MUTIL_CFLAGS \
MNETUTIL_LIBS=”$LOC_MNETUTIL_LIBS -lmnetutil” \
MNETUTIL_CFLAGS=$LOC_MNETUTIL_CFLAGS \
CPPFLAGS=”-I$prefix_cross_include ” \
CXXFLAGS=”-Wall $compiler_debug” \
LDFLAGS=”-L$prefix_cross_lib” \
./configure $host_option $build_option prefix=$prefix_cross $configure_params
fi
if [ ${subdir} = "libzrtpcpp" ]; then
LOC_ZRTPCPP_LIBS=-L$PWD/.libs
LOC_ZRTPCPP_CFLAGS=-I$PWD/include
MUTIL_LIBS=”$LOC_MUTIL_LIBS -lmutil” \
MUTIL_CFLAGS=$LOC_MUTIL_CFLAGS \
MNETUTIL_LIBS=”$LOC_MNETUTIL_LIBS -lmnetutil” \
MNETUTIL_CFLAGS=$LOC_MNETUTIL_CFLAGS \
CPPFLAGS=”-I$prefix_cross_include ” \
CXXFLAGS=”-Wall $compiler_debug” \
LDFLAGS=”-L$prefix_cross_lib” \
./configure $host_option $build_option prefix=$prefix_cross $configure_params
fi
if [ ${subdir} = "libminisip" ] ; then
LOC_LIBMINISIP_LIBS=-L$PWD/.libs
LOC_LIBMINISIP_CFLAGS=-I$PWD/include
configure_params_libminisip=”$configure_params $libminisip_configure_params”
echo libminisip can also have special config params
configure_params=”$configure_params $minisip_configure_params”
MUTIL_LIBS=”$LOC_MUTIL_LIBS -lmutil” \
MUTIL_CFLAGS=$LOC_MUTIL_CFLAGS \
MNETUTIL_LIBS=”$LOC_MNETUTIL_LIBS -lmnetutil” \
MNETUTIL_CFLAGS=$LOC_MNETUTIL_CFLAGS \
MIKEY_LIBS=”$LOC_MIKEY_LIBS -lmikey” \
MIKEY_CFLAGS=$LOC_MIKEY_CFLAGS \
MSIP_LIBS=”$LOC_MSIP_LIBS -lmsip” \
MSIP_CFLAGS=$LOC_MSIP_CFLAGS \
CPPFLAGS=”-I$prefix_cross_include ” \
CXXFLAGS=”-Wall $compiler_debug” \
LDFLAGS=”-L$prefix_cross_lib” \
./configure $host_option $build_option prefix=$prefix_cross $configure_params_libminisip
fi
if [ ${subdir} = "minisip" ]; then
echo minisip can also have special config params
configure_params_minisip=”$configure_params $minisip_configure_params”
MUTIL_LIBS=”$LOC_MUTIL_LIBS -lmutil” \
MUTIL_CFLAGS=$LOC_MUTIL_CFLAGS \
MNETUTIL_LIBS=”$LOC_MNETUTIL_LIBS -lmnetutil” \
MNETUTIL_CFLAGS=$LOC_MNETUTIL_CFLAGS \
MIKEY_LIBS=”$LOC_MIKEY_LIBS -lmikey” \
MIKEY_CFLAGS=$LOC_MIKEY_CFLAGS \
MSIP_LIBS=”$LOC_MSIP_LIBS -lmsip” \
MSIP_CFLAGS=$LOC_MSIP_CFLAGS \
LIBMINISIP_LIBS=”$LOC_LIBMINISIP_LIBS -lminisip” \
LIBMINISIP_CFLAGS=$LOC_LIBMINISIP_CFLAGS \
CPPFLAGS=”-I$prefix_cross_include ” \
CXXFLAGS=”-Wall $compiler_debug” \
LDFLAGS=”-L$prefix_cross_lib” \
./configure $host_option $build_option prefix=$prefix_cross $configure_params_minisip
fi
echo “==========================================================”
echo “configure_params (${subdir})= $configure_params”
echo “==========================================================”
make clean
make
make install
export PKG_CONFIG_PATH=$prefix_cross/lib/pkgconfig/
echo $prefix_cross/share/aclocal >> /usr/share/aclocal/dirlist
echo $prefix_cross/share/aclocal >> /usr/local/share/aclocal/dirlist
cd ..
done
mkdir -p compiled_files
mkdir -p compiled_files/bin
mkdir -p compiled_files/lib
mkdir -p compiled_files/share
mkdir -p compiled_files/share/minisip
mkdir -p compiled_files/lib/libminisip
mkdir -p compiled_files/lib/libminisip/plugins
tmp_lib=”libmutil”
cp -f $tmp_lib/.libs/$tmp_lib-0.dll compiled_files/bin
tmp_lib=”libmnetutil”
cp -f $tmp_lib/.libs/$tmp_lib-0.dll compiled_files/bin
tmp_lib=”libmcrypto”
cp -f $tmp_lib/.libs/$tmp_lib-0.dll compiled_files/bin
tmp_lib=”libmikey”
cp -f $tmp_lib/.libs/$tmp_lib-0.dll compiled_files/bin
tmp_lib=”libzrtpcpp”
cp -f $tmp_lib/.libs/$tmp_lib-0.dll compiled_files/bin
tmp_lib=”libmsip”
cp -f $tmp_lib/.libs/$tmp_lib-0.dll compiled_files/bin
tmp_lib=”libmstun”
cp -f $tmp_lib/.libs/$tmp_lib-0.dll compiled_files/bin
tmp_lib=”libminisip”
cp -f $tmp_lib/.libs/$tmp_lib-0.dll compiled_files/bin
cp -f $tmp_lib/.libs/mdsound.dll compiled_files/lib/libminisip/plugins
cp -f $tmp_lib/.libs/mdsound.la compiled_files/lib/libminisip/plugins
cp -f minisip/minisip/gui/gtkgui/minisip_gtkgui.exe compiled_files/bin
cp -f $prefix_cross/lib/libcrypto.dll compiled_files/bin/libeay32.dll
cp -f $prefix_cross/lib/libssl.dll compiled_files/bin/ssleay32.dll
cp -f $prefix_cross/share/minisip/* compiled_files/share/minisip
find minisip/ -name minisip_gtkgui.exe -size +200k -exec cp -f {} compilied_files/ \;
echo 3
#find minisip/ -name minisip_textui.exe -size +200k -exec cp -f {} compilied_files/ \;
$strip_bin -R compiled_files/*

Then, once the SDK installed, go into the start menu, SDK, and find a x64 retail command line. From there, go into numpy sources, and build with python setup.py install – it should work.

A package is related to how a distribution is organized (where and how to put configuration files, where to put libraries, dependencies names and versions), so you would have to enforce the same organization, dependencies, for each distribution. Once you enforce this, you end up forcing every distribution to be almost exactly the same.