how to discrete the state
See original GitHub issueHi,Cihan Ceyhan. I have learned your code of flappybird-qlearning-bot, it’s excellent. But I have one questions about these codes, could you help me?
how to discrete the state? ` def map_state(self, xdif, ydif, vel): ‘’’ Map the (xdif, ydif, vel) to the respective state, with regards to the grids The state is a string, “xdif_ydif_vel”
X -> [-40,-30...120] U [140, 210 ... 420]
Y -> [-300, -290 ... 160] U [180, 240 ... 420]
'''
if xdif < 140:
xdif = int(xdif) - (int(xdif) % 10)
else:
xdif = int(xdif) - (int(xdif) % 70)
if ydif < 180:
ydif = int(ydif) - (int(ydif) % 10)
else:
ydif = int(ydif) - (int(ydif) % 60)
return str(int(xdif))+'_'+str(int(ydif))+'_'+str(vel)`
how do you determine the boundary of X and Y? Thank you very much.
Issue Analytics
- State:
- Created 5 years ago
- Comments:7 (3 by maintainers)
Top Results From Across the Web
State space discretization - Mathematics Stack Exchange
I'm playing with a continuous state space system and I have to discretize it. I've looked around and it seems pretty easy:.
Read more >Discrete-Time State Space Analysis
In this section, we show how to obtain discrete-time state space models from continuous-time system models. Integral Approximation Method. The integral ...
Read more >Discretization of Continuous Time State Space Systems - WPI
Suppose we are given the continuous time state space system ... and apply an input that changes only at discrete (equal) sampling intervals....
Read more >Discretization - Wikipedia
In applied mathematics, discretization is the process of transferring continuous functions, models, variables, and equations into discrete counterparts. ... Discretization of linear state...
Read more >Discrete State - an overview | ScienceDirect Topics
The discrete state amplitude is expressed in terms of a finite integral that can be evaluated numerically plus contributions from at most two...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
@chncyhn Thanks for your reply, now I have a deeper understanding. I will read the 'comparison of Q-learning vs SARSA ’ carefully, thanks for your help again, that means a lot for me .
@ProQianXiao yes your understanding is mostly correct. (except the “…no matter what action it took” statement, because we punish the (state,action) pairs, not just the state; note that we update
self.qvalues[state][act]
).Another small difference is that you said “… it means that the bird crashed on the upper pipe, so the states before crashing that took ‘flap’ action should be punished”, but only the last state where ‘flap’ action is taken is punished, specifically the last (state, action) pair where action is ‘flap’.
I haven’t looked much into it, but in this thread there is a good comparison of Q-learning vs SARSA which you might find useful.