Insertions are not in constant time

The documentation states that element insertion and deletion are in O(1). However, a quick benchmark using JMH reveals linear behaviour for adding vertices and edges.

[info] Benchmark                                                     (siz)  Mode  Cnt   Score   Error  Units
[info] LocalBenchmarks.List.ListBenchmark.addEdgeToImmutableGraph        0  avgt    2   0.017          ms/op
[info] LocalBenchmarks.List.ListBenchmark.addEdgeToImmutableGraph      100  avgt    2   0.030          ms/op
[info] LocalBenchmarks.List.ListBenchmark.addEdgeToImmutableGraph      500  avgt    2   0.094          ms/op
[info] LocalBenchmarks.List.ListBenchmark.addEdgeToImmutableGraph     1000  avgt    2   0.150          ms/op
[info] LocalBenchmarks.List.ListBenchmark.addEdgeToMutableGraph          0  avgt    2   0.017          ms/op
[info] LocalBenchmarks.List.ListBenchmark.addEdgeToMutableGraph        100  avgt    2   0.027          ms/op
[info] LocalBenchmarks.List.ListBenchmark.addEdgeToMutableGraph        500  avgt    2   0.068          ms/op
[info] LocalBenchmarks.List.ListBenchmark.addEdgeToMutableGraph       1000  avgt    2   0.118          ms/op
[info] LocalBenchmarks.List.ListBenchmark.addVertexToImmutableGraph      0  avgt    2  ≈ 10⁻⁴          ms/op
[info] LocalBenchmarks.List.ListBenchmark.addVertexToImmutableGraph    100  avgt    2   0.013          ms/op
[info] LocalBenchmarks.List.ListBenchmark.addVertexToImmutableGraph    500  avgt    2   0.076          ms/op
[info] LocalBenchmarks.List.ListBenchmark.addVertexToImmutableGraph   1000  avgt    2   0.170          ms/op
[info] LocalBenchmarks.List.ListBenchmark.addVertexToMutableGraph        0  avgt    2  ≈ 10⁻⁴          ms/op
[info] LocalBenchmarks.List.ListBenchmark.addVertexToMutableGraph      100  avgt    2   0.018          ms/op
[info] LocalBenchmarks.List.ListBenchmark.addVertexToMutableGraph      500  avgt    2   0.110          ms/op
[info] LocalBenchmarks.List.ListBenchmark.addVertexToMutableGraph     1000  avgt    2   0.254          ms/op
[success] Total time: 497 s (08:17), completed Mar 20, 2020 2:51:04 PM

As you can see, the execution time increases when the size of the graph (the number of vertices) increases. I didn’t test the effect of the number of edges on the execution time.

I don’t think we should change the documentation to say O(n) but rather fix the implementation such that it is indeed in constant time. However, I don’t know what causes this linear behaviour.

---

The benchmarking code I used:

import org.openjdk.jmh.annotations._
import scalax.collection.GraphEdge.DiEdge
import scalax.collection.GraphPredef.EdgeAssoc

@State(Scope.Benchmark)
class GraphState {
  import scalax.collection.Graph

  @Param(Array("0", "100", "500", "1000"))
  var siz: Int = 0

  var graph: Graph[String, DiEdge] = _

  @Setup(Level.Trial)
  def setup(): Unit = {
    graph = Graph.empty[String, DiEdge]
    (1 to siz)
      .foreach(i => graph = graph + s"Vertex-$i")
  }
}

@State(Scope.Benchmark)
class MutableGraphState {
  import scalax.collection.mutable.Graph

  @Param(Array("0", "100", "500", "1000"))
  var siz: Int = 0

  var graph: Graph[String, DiEdge] = _

  @Setup(Level.Invocation)
  def setup(): Unit = {
    graph = Graph.empty[String, DiEdge]
    (1 to siz)
      .foreach(i => graph += s"Vertex-$i")
    println(s"graph size: ${graph.size}")
  }
}

@BenchmarkMode(Array(Mode.AverageTime))
@OutputTimeUnit(TimeUnit.MILLISECONDS)
class GraphBenchmark {
  @Benchmark
  def addVertexToImmutableGraph(state: GraphState): scalax.collection.Graph[String, DiEdge] = {
    state.graph + "vertex"
  }

  @Benchmark
  def addEdgeToImmutableGraph(state: GraphState): scalax.collection.Graph[String, DiEdge] = {
    state.graph + ("Vertex-1" ~> "Vertex-2")
  }

  @Benchmark
  def addVertexToMutableGraph(state: MutableGraphState): scalax.collection.mutable.Graph[String, DiEdge] = {
    state.graph + "vertex"
  }

  @Benchmark
  def addEdgeToMutableGraph(state: MutableGraphState): scalax.collection.mutable.Graph[String, DiEdge] = {
    state.graph + ("Vertex-1" ~> "Vertex-2")
  }
}

To run a quick benchmark:

sbt
jmh:run -wi 1 -i 2 -f1 -t1

To run a more precise benchmark: jmh:run

[peter-empen]

As with Scala 2.12 collections, + creates a new instance while += does not. So when benchmarking mutable graphs you should not use the + operator.

[peter-empen]

Documentation fixed. Hope your benchmark now reflects the documented characteristics.