This thesis investigates the application of Physics-Informed Neural Networks(PINNs) to solve Hamilton-Jacobi-Bellman (HJB) equations, arising incontinuous time Reinforcement Learning (RL) and optimal control problems.HJB equations, derived from dynamic programming, are essential fordetermining optimal policies but are challenging to solve as they are nonlinearpartial differential equations (PDEs). Traditional numerical methods, such asfinite element methods, often face scalability and efficiency issues in highdimensionalcontexts. PINNs leverage the power of deep learning by embeddingthe physical laws governing these equations directly into the neural networktraining process. Recent advancements in deep learning have shown that PINNscan effectively solve complex PDEs. This thesis aims to implement and evaluatePINNs for efficiently solving HJB equations, offering a scalable and accurateframework for optimal decision-making in continuous time RL and controlscenarios.