process_function in Engine has strong signature?
See original GitHub issueI checked all docs, but I can’t understand if we have any restriction for process_function
for Engine
.
For example, I want to resolve the following questions:
- Must my
process_function
hasengine
as a first argument? If it is true, then it would be great to notice about it in docs. - Can I use another signature instead of
process_function(engine, labelled_batch)
? For instance,process_function(engine, labelled_batch, device)
? According to this great code, it is used additional function_prepare_batch
, because of strong signature ofprocess_function
(otherwise, I would passdevice
toprocess_function
as third argument).
Issue Analytics
- State:
- Created 4 years ago
- Comments:7 (1 by maintainers)
Top Results From Across the Web
Cannot extend Flink ProcessFunction - scala - Stack Overflow
I recently upgraded from Flink 1.2 to Flink 1.3, and I am trying to update my ProcessFunction to work with 1.3. I have...
Read more >ProcessFunction (Flink : 1.16-SNAPSHOT API)
A function that processes elements of a stream. For every element in the input stream processElement(Object, Context, Collector) is invoked.
Read more >State Management in Apache Flink - DataDome
If you're fine with losing the state of a given operator, ... going to define the signature for a process-function depending on its...
Read more >Scripting (continued) - Godot Docs
Its execution is done after the physics step on single-threaded games. A simple way to see the _process() function at work is to...
Read more >Usage — AiiDA 2.2.0 documentation
For process functions, the ProcessSpec is dynamically generated by the engine from the signature of the decorated function. Therefore, to determine what inputs ......
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@ZhiliangWu having engine as an argument of
process_function
can help with fetching easily the current state: iteration, epoch, custom variables etc. For example, when during the training you have total loss function composed of two part and an alpha parameter that is also updated during the training:This is just an example, but you can imagine other use-cases where usage of trainer’s state can be interesting.
Hi @vfdev-5, I am trying to use this great package to accelerate my development cycle. However, I cannot quite understand why the function
process_function()
must takeengine
as its input argument. The only usage of it is in the_run_once_on_dataset()
method to compute theself.state.output
. However, the argumentengine
(orself
when called in the class) doesn’t do anything or save anything with it. Are there any deeper design thoughts on it? Or is it only used to show deliberately that the process is associated with a specific engine? Thanks for your comments in advance.