minizinc solver: how to get all solutions fast

Here is a program which compares three different solvers on a simple model:

from cpmpy import *
from cpmpy.solvers import *
from cpmpy.solvers.utils import get_supported_solvers
import numpy as np
import time

def get_different_solution(m,x):
  """
  get_different_solution(m,x)
  
  Add current solution to the model m for generating other solutions.
  """
  m += [any([t.value() != t for t in x])]

def increasing(args):
  """
  Ensure that the values in args are increasing.
  """
  return [args[i-1] <= args[i] for i in range(1,len(args))]


def test2(solver):
  print("solver:",solver)
  
  n = 3


  x = intvar(0,n, shape=n,name="x")
  z = intvar(0,n*n,name="z")
  print("x:",x)

  model = Model([
    # increasing(x),
    
    # AllDifferent_except_0(x), # same error
    

    # This works but is slow
    AllDifferent(x),
  
    # This works but is extremely slow!
    # z == sum(x),
    z == x.sum(),    
    ])
  
  print("model:", model)
  num_solutions = 0

  if solver == "ortools":
   # solver with ortools
    t1 = time.time()
    ss = CPM_ortools(model)
    while ss.solve():
      num_solutions += 1
      print("x:",x.value(),"z:",z.value(),flush=True)
      get_different_solution(ss,x)    
    print("num_solutions:", num_solutions)
    t2 = time.time()
    print("ortools time:", t2-t1)
    print()
    
  elif solver == "minizinc":
    # Solve with minizinc
    t3 = time.time()
    ss = CPM_minizinc(model)
    # ss = CPM_minizinc(model,"or_tools") # Setting to a specific don't seems to matter. See below
    while ss.solve():
      num_solutions += 1
      print("x:",x.value(),"z:",z.value(),flush=True)
      get_different_solution(ss,x)
      t4 = time.time()
    print("minizinc time:", t4-t3)
    
  else :
    # Default solver
    t3 = time.time()
    while model.solve():
      num_solutions += 1
      print("x:",x.value(),"z:",z.value(),flush=True)
      get_different_solution(model,x)
      t4 = time.time()
    print("minizinc time:", t4-t3)
  
test("ortools")
test("minizinc")
test("default")

The time for the MiniZinc approach is significantly slower than the two other approaches:

ortools time: 0.05745697021484375
minizinc time: 5.6118950843811035
default time: 0.13840413093566895

The file minizinc.py mention all_solutions=True as a parameter which I assume should be used like this:

   # ...
    ss = CPM_minizinc(model)
    while ss.solve(**all_solutions=True)**:
    # ...

But then the output is not correct:

Warning, no value for  x_0
Warning, no value for  x_1
Warning, no value for  x_2
Warning, no value for  z
x: [None None None] z: None

Note that it don’t matter which FlatZinc solver that is used. I tested with some different solvers such as gecode, chuffed, or_tools, picat_sat (using ss = CPM_minizinc(model,"")) but there wasn’t any speedup.

What have I missed?

[tias]

We’ve implemented a generic version, with the more efficient solver-specifc one for ortools and MiniZinc (which was subtle to get right).

It is master, we hope to do a new beta release with the unified solver interfaces in as well before the end of the week, let’s see : )

So, issue closed and I’m very happy with solveAll() as a new inference method next to solve()

Thanks for the discussions Hakan!

[tias]

I reverted it from main and created a new branch, as so many aspects of CPMpy it seems like it can benefit from some more thought and a uniform implementation.

About the name, I think solveAll is appropriate for the query, as would solveCount be, but ‘all solution search’ is a more common description hence my preference for solveAll I think…

I see it as a query, which should return the count so that you can easily use it in meta-algorithms. For me, the displaying/printing is just a convenience on top.

That said, I do agree/realize that the ‘variables’ argument gives the impression that it influences what happens in the function (e.g. a projection) which is not at all the case. Also that it could be made even more convenient, namely to accept either nothing, a (list of) arbitrary CPMpy expressions (including, e.g. an objective function expression) or a python callback function. I refactord in that that way, example uses:

x = intvar(0,3, shape=2)
m = Model(x[0] > x[1])
s = CPM_ortools(m)

s.solveAll()
s.solveAll(display=x)
s.solveAll(display=[x,sum(x)])

def newprint():
    print(f"x0:{x[0].value()}")
s.solveAll(display=newprint)

solutions = []
def collect():
    print(x.value())
    solutions.append(x.value())
s.solveAll(display=collect)

print(solutions)

the latter is possible because Python actually has very flexible closure mechanisms on its functions…

Another simplification done in the above is that there is just one argument ‘display’ both for lists/expressions and a callback…

Then, finally, about the solution limit; we also have time_limit, so I guess it makes sense:

s.solveAll(display=[x, sum(x)], solution_limit=3)

I think this is pretty neat and covers our discussions?

I would then preferably make this an API call of Model() and all solver-specific interfaces (with as fallback the blocking-clause version if the solver has no native support). + docs etc…