Running 200000 Instances of a single POST call causes JS exception

Version and environment information:

1. Newman Version (can be found via newman -v): 3.4.3
2. OS details (type, version, and architecture): RHEL - Red Hat Enterprise Linux
3. Are you using Newman as a library, or via the CLI? CLI
4. Did you encounter this recently, or has this bug always been there: Presumably always there. Ran into this recently
5. Expected behaviour: There be better garbage collection/cleanup and I am able to run all 200K instances
6. Command / script used to run Newman: _(From Jenkins Job): /usr/local/lib/node_modules/newman/bin/newman.js run ${COLLECTION_NAME} -e ${ENVIRONMENT_NAME} -d serverInfo.csv -n ${COUNT} -r html,cli,json,junit --insecure _
7. Sample collection, and auxilliary files (minus the sensitive details): N/A
8. Screenshots (if applicable): No Screen shots but exception below Exception:

Iteration 144112/200000

→ Start A Workflow

<--- Last few GCs --->

[8724:0x3ea00d0] 294323968 ms: Mark-sweep 1213.5 (1553.3) -> 1213.3 (1553.3) MB, 1790.0 / 0.0 ms  allocation failure GC in old space requested
[8724:0x3ea00d0] 294325847 ms: Mark-sweep 1213.3 (1553.3) -> 1213.2 (1513.3) MB, 1841.3 / 0.0 ms  last resort gc 
[8724:0x3ea00d0] 294327659 ms: Mark-sweep 1213.2 (1513.3) -> 1213.2 (1506.3) MB, 1812.1 / 0.0 ms  last resort gc 


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x123a3f9c0d31 <JS Object>
    1: /* anonymous */ [/usr/local/lib/node_modules/newman/node_modules/postman-collection/lib/collection/property-list.js:359] [pc=0xa7418cac6e4](this=0x84092ea5e21 <JS Global Object>,accumulator=0x187d00a26dc9 <an Object with map 0x7ce66607011>,value=0x2d0ef3df7ce1 <a PostmanPropertyDescription with map 0x26850f316d69>,key=0x84092e69769 <String[11]: description>)
    2: arguments adaptor fram...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x126492c [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewTransitionArray(int) [node]
 6: v8::internal::TransitionArray::Insert(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Map>, v8::internal::SimpleTransitionFlag) [node]
 7: v8::internal::Map::CopyReplaceDescriptors(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::DescriptorArray>, v8::internal::Handle<v8::internal::LayoutDescriptor>, v8::internal::TransitionFlag, v8::internal::MaybeHandle<v8::internal::Name>, char const*, v8::internal::SimpleTransitionFlag) [node]
 8: v8::internal::Map::CopyAddDescriptor(v8::internal::Handle<v8::internal::Map>, v8::internal::Descriptor*, v8::internal::TransitionFlag) [node]
 9: v8::internal::Map::CopyWithField(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::FieldType>, v8::internal::PropertyAttributes, v8::internal::Representation, v8::internal::TransitionFlag) [node]
10: v8::internal::Map::TransitionToDataProperty(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::StoreFromKeyed) [node]
11: v8::internal::LookupIterator::PrepareTransitionToDataProperty(v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::StoreFromKeyed) [node]
12: v8::internal::Object::AddDataProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::ShouldThrow, v8::internal::Object::StoreFromKeyed) [node]
13: v8::internal::Object::SetProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::LanguageMode, v8::internal::Object::StoreFromKeyed) [node]
14: v8::internal::StoreIC::Store(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::Object::StoreFromKeyed) [node]
15: v8::internal::KeyedStoreIC::Store(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [node]
16: v8::internal::Runtime_KeyedStoreIC_Miss(int, v8::internal::Object**, v8::internal::Isolate*) [node]
17: 0xa74174063a7
/tmp/hudson5071886919626673712.sh: line 9:  8724 Aborted                 /usr/local/lib/node_modules/newman/bin/newman.js run ${COLLECTION_NAME} -e ${ENVIRONMENT_NAME} -d serverInfo.csv -n ${COUNT} -r html,cli,json,junit --insecure

Steps to reproduce the problem:

1. Run newman with 200K instances of a single call.

[shamasis]

@kunagpal @kamalaknn we need to build a newman core summary object flag that does not retain / accumulate stuff at all.

Else, we need to check if turning off all reporters (including CLI) ensures a light operation. If that is done, then we can write an article on turning off the reporters and maybe using a light reporter to run many many requests.

[shamasis]

We need to decouple the “accumulator” summary object for reporters in Newman. It will disable some statistics feature, but it should open up this use case.

@codenirvana how about we do a workaround as an interim that when no reporter is loaded, we don’t build summary object?

Then we can schedule a reporter API change for next major newman version or back the summary object with a disk buffer?

We would also need to do something about the extractRunnables array that flattens a collection and instead use a DSL like approach to programmatically track outcome for large collections. But, I’m guessing that for single request and large iteration use case, even this will not be a blocker.