https://www.nikhilkarnwal.com/ai-lab daily 0.75 2026-04-12 https://www.nikhilkarnwal.com/ai-lab/part-1-imitation-learning-amp-why-it-breaks monthly 0.5 2026-04-12 https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/1e4127ef-fe93-4430-81aa-9b0437ac3442/rl_agent_loop.png AI-Lab - Part 1: Imitation Learning & Why It Breaks - Make it stand out Figure 1: The Reinforcement Learning loop. At each time step, the agent observes the current state s_t, from the environment, takes an action a_t, and in return receives a new state and a reward r_t. https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/e8c6f25f-9c40-4477-a125-9701d318950c/ant_v5_expert.gif AI-Lab - Part 1: Imitation Learning & Why It Breaks - Make it stand out Figure 2: The expert SAC policy walking the Ant-v5 robot. This is our gold standard — the behavior we want to imitate. https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/d583fb8c-b89d-4c52-8e68-006c1455bc11/ant_v5_bc_loss.png AI-Lab - Part 1: Imitation Learning & Why It Breaks - Make it stand out BC training loss (MSE) dropping over 100 epochs. Lower means the policy's predictions are closer to the expert's actions. https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/9eb653c5-c8df-4c54-a39b-c7ea4fb90134/ant_v5_bc_d_shift.png AI-Lab - Part 1: Imitation Learning & Why It Breaks - Make it stand out Expert is maintaining stable height around 0.6(lower compare to BC) and able to walk faster compare to BC. https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/be6bedbc-e832-4c71-b45d-2be2cff236f1/ant_v5_bc_dagger_loss.png AI-Lab - Part 1: Imitation Learning & Why It Breaks - Make it stand out Full comparison: DAgger progressively closes the gap with the expert as it aggregates more data. https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/7afb55f0-c327-40f3-bf76-e54fe3bcfc8d/ant_v5_comp_render.gif AI-Lab - Part 1: Imitation Learning & Why It Breaks - Make it stand out Whatever it is, the way you tell your story online can make all the difference. https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/d7ee288f-e078-4598-bb6b-4f8befff6b8f/ant_v5_bc_dagger_d_shift.png AI-Lab - Part 1: Imitation Learning & Why It Breaks - Make it stand out DAgger closes the distribution gap. Its state distribution (purple) is much closer to the expert's (blue) than BC's was (red). https://www.nikhilkarnwal.com/home daily 1.0 2025-12-29 https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/13bfd477-8111-4032-b509-8c217787454e/nikhil_profile_photo.jpg https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/ad8aec5a-20c5-4beb-8c7f-e77bc14cccb4/IRL_PAPER_NIKHIL.png https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/c3261967-430c-4a43-af9b-ac97b083980b/bio_bert_NIKHIL_PROJECT.jpg https://www.nikhilkarnwal.com/about daily 0.75 2025-12-28 https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/f3a1e545-a02f-4608-9846-3464585d1c47/ChatGPT+Image+Dec+27%2C+2025%2C+03_28_04+PM.png https://www.nikhilkarnwal.com/contact daily 0.75 2025-12-28 https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/1722380712.062739-NUFNWVXVOFAISIAECDXX/imgg-od3-lauctej2.png https://www.nikhilkarnwal.com/projects daily 0.75 2025-12-28 https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/83e31a87-afd8-4668-bb85-26464074f111/IRL_PAPER_NIKHIL.png https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/76784207-62cf-4ac1-bfe5-c4c51f497014/rl_g_football_nikhil_project.png https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/03f5a013-97dd-443f-b0ad-e80d12c11cad/vis_ground_nikhil_project.png https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/c559de01-adab-4234-a5bf-0b674fcb3075/in_cabin_nikhil_project.png https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/fd30a8a5-d5ae-41d8-b708-a3219fbcf3e4/gan_video_comp_nikhil_project.png https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/593205a6-f7de-4005-85ac-9a8a8bd51808/bio_bert_NIKHIL_PROJECT.jpg https://images.squarespace-cdn.com/content/v1/694db376617bc461a02126ae/a62739ba-503c-4f06-b511-5502e707ef73/bio_cancer_nikhil_project.png