Code secret: 395800<\/em><\/p>\n\n\n\nMembres du jury :<\/strong><\/p>\n\n\n\nM. Djamal zEGHLACHE<\/strong>, Professeur, T\u00e9l\u00e9com SudParis, FRANCE – Directeur de these
M. Khaled BOUSSETTA<\/strong>, Professeur, Paris 13, FRANCE – Rapporteur<\/p>\n\n\n\nM. Salah-Eddine EL AYOUBI, Professeur, CentraleSup\u00e9lec, FRANCE – Rapporteur<\/strong><\/p>\n\n\n\nM. Pascal LORENZ, Professeur, Universit\u00e9 <\/strong>de Haute Alsace, FRANCE – Examinateur<\/strong><\/p>\n\n\n\nM. Hadji MAKHLOUF, Chercheur Scientific, Institut de Recherch Technologique SystemX, FRANCE – Examinateur<\/strong><\/p>\n\n\n\nMme. Thi-Mai-Trang NGUYEN, Professeur, Universit\u00e9 Sorbonne Paris Nord, FRANCE – Examinatrice<\/strong><\/p>\n\n\n\nMme. Rola NAJA, HDR et Associate Professor, ECE Paris Lyon – Ecole d’ing\u00e9nieurs, FRANCE – Co-encadrant<\/strong><\/p>\n\n\n\nInvit\u00e9 :<\/p>\n\n\n\n
M. Samir TOHME, Professeur, Universit\u00e9 de Versailles Saint-Quentin-en-Yvelines, FRANCE<\/strong><\/p>\n\n\n\n\u00ab APPRENTISSAGE POUR LA SURETE DANS LES RESEAUX VEHICULAIRES \u00bb<\/h2>\n\n\n\npr\u00e9sent\u00e9 par Madame Jialin HAO<\/h2>\n\n\n\n
R\u00e9sum\u00e9 :<\/strong><\/p>\n\n\n\nCette th\u00e8se porte sur le d\u00e9veloppement d’une man\u0153uvre d’aide au changement de voie (lane Change Assistance, LCA) s\u00fbre et efficace dans le contexte des r\u00e9seaux de v\u00e9hicules assist\u00e9s par drones (Drone Assisted Vehicular Network, DAVN). En effet, les changements de voie contribuent de mani\u00e8re significative aux accidents de la route, n\u00e9cessitant des solutions efficaces au sein des r\u00e9seaux routiers. Les LCA strat\u00e9gies actuelles \u00e9tablies sur l’apprentissage par renforcement profond (Deep Reinforcement Learning, DRL) sont limit\u00e9es par les informations locales sur les v\u00e9hicules, n\u00e9gligeant une vue globale, comme des conditions de circulation. Pour r\u00e9soudre ce probl\u00e8me, les v\u00e9hicules a\u00e9riens sans pilote (Unmanned Aerial Vehicles, UAVs), ou drones, pr\u00e9sentent une extension prometteuse des services de r\u00e9seau automobile gr\u00e2ce \u00e0 leur mobilit\u00e9, capacit\u00e9s informatiques et liaisons de communication en visibilit\u00e9 directe (Line-if-Sight, LoS) avec les v\u00e9hicules routiers. Dans un premier temps, nous faisons une \u00e9tude bibliographique sur LCA au sein du DAVN, mettant en \u00e9vidence le potentiel des drones pour am\u00e9liorer la s\u00e9curit\u00e9 routi\u00e8re. Les approches LCA existantes s’appuient principalement sur des informations locales sur les v\u00e9hicules et ne prennent pas en compte l’\u00e9tat global du trafic. Afin de r\u00e9duire cette limitation, nous proposons le GL-DEAR : joint global and local drone-assisted lane change platform based on Deep-Q Network (DQN) with a dynamic reward function, for LCA with drones’ assistance. La plateforme propos\u00e9e se compose de trois modules : route \u00e0 risques al\u00e9atoires et v\u00e9hicules d’urgence ; acquisition et traitement des donn\u00e9es\u202f; prise de d\u00e9cision de changement de voie en temps r\u00e9el. La man\u0153uvre de changement de voie est bas\u00e9e sur un Deep Q-Network avec des fonctions de r\u00e9compense dynamiques. Plus pr\u00e9cis\u00e9ment, nous adoptons les mod\u00e8les de changement de voie authentiques bas\u00e9s sur l’ensemble de donn\u00e9es NGSIM pour les v\u00e9hicules routiers ordinaires afin de recr\u00e9er les comportements de changement de voie du monde r\u00e9el dans les simulations. Les r\u00e9sultats num\u00e9riques d\u00e9montrent la capacit\u00e9 de la plateforme \u00e0 r\u00e9aliser des trajets sans collision sur des autoroutes \u00e0 risque avec des v\u00e9hicules d’urgence. Dans un deuxi\u00e8me temps, nous identifions un manque de calibrage de la fr\u00e9quence de mise \u00e0 jour globale des algorithmes d’apprentissage f\u00e9d\u00e9r\u00e9 (Federated Learning, FL) et l\u2019absence d\u2019\u00e9valuation approfondie du d\u00e9lai de traitement au niveau du drone. Nous proposons donc un cadre d’apprentissage par renforcement f\u00e9d\u00e9r\u00e9 (FRL) assist\u00e9 par drone, DAFL. Ce cadre permet un apprentissage coop\u00e9ratif entre les v\u00e9hicules de l’ego en appliquant FL. Il comprend un algorithme d’agr\u00e9gation de mod\u00e8les global bas\u00e9 sur la r\u00e9putation du client et une analyse compl\u00e8te du d\u00e9lai de bout en bout (End-to-End, E2E) au niveau du drone. Plus pr\u00e9cis\u00e9ment, la fr\u00e9quence globale de mise \u00e0 jour est ajust\u00e9e dynamiquement en fonction des mesures de s\u00e9curit\u00e9 routi\u00e8re et de la consommation \u00e9nerg\u00e9tique des drones, ce qui donne des r\u00e9sultats efficaces dans les simulations. Dans la troisi\u00e8me \u00e9tape, nous concevons l’algorithme DOP-T pour optimiser les trajectoires des drones dans les r\u00e9seaux de v\u00e9hicules dynamiques. Cet algorithme vise \u00e0 \u00e9quilibrer la consommation \u00e9nerg\u00e9tique des drones et la s\u00e9curit\u00e9 routi\u00e8re. Nous fournissons un \u00e9tat de l\u2019art complet des techniques existantes de planification de trajectoire de drones. Ensuite, sur la base de la mod\u00e9lisation du d\u00e9lai E2E du v\u00e9hicule et de la mod\u00e9lisation de la consommation d’\u00e9nergie du drone. Dans la seconde \u00e9tape, nous formons un mod\u00e8le d’apprentissage par renforcement hors ligne (Offline-Reinforcement Learning, ORL) pour \u00e9viter une formation en ligne consommatrice d’\u00e9nergie. Les r\u00e9sultats de la simulation d\u00e9montrent une r\u00e9duction significative de la consommation d’\u00e9nergie des drones et du d\u00e9lai E2E du v\u00e9hicule \u00e0 l’aide du mod\u00e8le entra\u00een\u00e9.
Abstract :<\/strong><\/p>\n\n\n\nThis thesis focuses on the development of a safe and efficient LCA maneuver in the context of drone-assisted vehicle networks (DAVN). In fact, lane change maneuvers contribute significantly to road accidents, requiring effective solutions within road networks. Current lane change assistance (LCA) strategies relying solely on deep reinforcement learning (DRL) are limited by local vehicle information, neglecting a global view of traffic conditions. To address this problem, unmanned aerial vehicles (UAVs), or drones, present a promising extension of automotive network services due to their mobility, computing capabilities, and line-of-sight (LoS) communications links with road vehicles. In the first step, we conduct a literature review on LCA within DAVN, highlighting the potential of drones to enhance road safety. Existing LCA approaches predominantly rely on local vehicle information and fail to consider overall traffic states. To address this limitation, we propose the GL-DEAR: joint global and local drone-assisted lane change platform based on Deep-Q Network (DQN) with a dynamic reward function, for LCA with drones’ assistance. The proposed platform consists of three modules: road with random risks and emergency vehicles; data file acquisition and processing; and real-time lane change decision-making. The lane change maneuver is based on a Deep Q-Network with dynamic reward functions. Specifically, we adopt the authentic NGSIM dataset-based lane change models for ordinary road vehicles to recreate real world lane change behaviors in the simulations. Numerical results demonstrate the platform’s ability to achieve collision-free trips on risky highways with emergency vehicles. In the second step, we identify a lack of calibration for the global update frequency in FL algorithms and the absence of thorough drone-level processing delay assessment. To this end, we propose the drone assisted Federated Reinforcement Learning (FRL)-based LCA framework, DAFL. This framework enables cooperative learning between ego vehicles by applying Federated Learning (FL). It includes a client reputation-based global model aggregation algorithm and a comprehensive analysis of End-to-End (E2E) delay at the drone. Specifically, the global update frequency is dynamically adjusted according to road safety measurements and drone energy consumption, yielding efficient results in simulations. In the third step, we devise the DOP-T algorithm for optimizing drone trajectories in dynamic vehicular networks. This algorithm aims to balance drone energy consumption and road safety. We provide a comprehensive state-of-the-art review of the existing drone trajectory planning techniques. Then, based on the vehicle E2E delay modeling and the drone energy consumption modeling in the second step, we train a Offline Reinforcement Learning (ORL) model to avoid power-consuming online training. Simulation results demonstrate a significant reduction in drone energy consumption and vehicle E2E delay using the trained model.<\/p>\n","protected":false},"excerpt":{"rendered":"
L’Ecole doctorale : Ecole Doctorale de l’Institut Polytechnique de Paris et le Laboratoire de recherche SAMOVAR – Services r\u00e9partis, Architectures, Mod\u00e9lisation, Validation, Administration des R\u00e9seaux pr\u00e9sentent l\u2019AVIS DE SOUTENANCE de Madame Jialin HAO Autoris\u00e9e \u00e0 pr\u00e9senter ses travaux en vue de l\u2019obtention du Doctorat de l’Institut Polytechnique de Paris, pr\u00e9par\u00e9 \u00e0 T\u00e9l\u00e9com SudParis en : […]<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ocean_post_layout":"","ocean_both_sidebars_style":"","ocean_both_sidebars_content_width":0,"ocean_both_sidebars_sidebars_width":0,"ocean_sidebar":"","ocean_second_sidebar":"","ocean_disable_margins":"enable","ocean_add_body_class":"","ocean_shortcode_before_top_bar":"","ocean_shortcode_after_top_bar":"","ocean_shortcode_before_header":"","ocean_shortcode_after_header":"","ocean_has_shortcode":"","ocean_shortcode_after_title":"","ocean_shortcode_before_footer_widgets":"","ocean_shortcode_after_footer_widgets":"","ocean_shortcode_before_footer_bottom":"","ocean_shortcode_after_footer_bottom":"","ocean_display_top_bar":"default","ocean_display_header":"default","ocean_header_style":"","ocean_center_header_left_menu":"","ocean_custom_header_template":"","ocean_custom_logo":0,"ocean_custom_retina_logo":0,"ocean_custom_logo_max_width":0,"ocean_custom_logo_tablet_max_width":0,"ocean_custom_logo_mobile_max_width":0,"ocean_custom_logo_max_height":0,"ocean_custom_logo_tablet_max_height":0,"ocean_custom_logo_mobile_max_height":0,"ocean_header_custom_menu":"","ocean_menu_typo_font_family":"","ocean_menu_typo_font_subset":"","ocean_menu_typo_font_size":0,"ocean_menu_typo_font_size_tablet":0,"ocean_menu_typo_font_size_mobile":0,"ocean_menu_typo_font_size_unit":"px","ocean_menu_typo_font_weight":"","ocean_menu_typo_font_weight_tablet":"","ocean_menu_typo_font_weight_mobile":"","ocean_menu_typo_transform":"","ocean_menu_typo_transform_tablet":"","ocean_menu_typo_transform_mobile":"","ocean_menu_typo_line_height":0,"ocean_menu_typo_line_height_tablet":0,"ocean_menu_typo_line_height_mobile":0,"ocean_menu_typo_line_height_unit":"","ocean_menu_typo_spacing":0,"ocean_menu_typo_spacing_tablet":0,"ocean_menu_typo_spacing_mobile":0,"ocean_menu_typo_spacing_unit":"","ocean_menu_link_color":"","ocean_menu_link_color_hover":"","ocean_menu_link_color_active":"","ocean_menu_link_background":"","ocean_menu_link_hover_background":"","ocean_menu_link_active_background":"","ocean_menu_social_links_bg":"","ocean_menu_social_hover_links_bg":"","ocean_menu_social_links_color":"","ocean_menu_social_hover_links_color":"","ocean_disable_title":"default","ocean_disable_heading":"default","ocean_post_title":"","ocean_post_subheading":"","ocean_post_title_style":"","ocean_post_title_background_color":"","ocean_post_title_background":0,"ocean_post_title_bg_image_position":"","ocean_post_title_bg_image_attachment":"","ocean_post_title_bg_image_repeat":"","ocean_post_title_bg_image_size":"","ocean_post_title_height":0,"ocean_post_title_bg_overlay":0.5,"ocean_post_title_bg_overlay_color":"","ocean_disable_breadcrumbs":"default","ocean_breadcrumbs_color":"","ocean_breadcrumbs_separator_color":"","ocean_breadcrumbs_links_color":"","ocean_breadcrumbs_links_hover_color":"","ocean_display_footer_widgets":"default","ocean_display_footer_bottom":"default","ocean_custom_footer_template":"","ocean_post_oembed":"","ocean_post_self_hosted_media":"","ocean_post_video_embed":"","ocean_link_format":"","ocean_link_format_target":"self","ocean_quote_format":"","ocean_quote_format_link":"post","ocean_gallery_link_images":"on","ocean_gallery_id":[],"footnotes":""},"categories":[286,402],"tags":[],"class_list":["post-6531","post","type-post","status-publish","format-standard","hentry","category-fractualites-ennews-fr","category-seminaires-ness-2013-fr","entry"],"_links":{"self":[{"href":"https:\/\/samovar.telecom-sudparis.eu\/index.php\/wp-json\/wp\/v2\/posts\/6531","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/samovar.telecom-sudparis.eu\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/samovar.telecom-sudparis.eu\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/samovar.telecom-sudparis.eu\/index.php\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/samovar.telecom-sudparis.eu\/index.php\/wp-json\/wp\/v2\/comments?post=6531"}],"version-history":[{"count":1,"href":"https:\/\/samovar.telecom-sudparis.eu\/index.php\/wp-json\/wp\/v2\/posts\/6531\/revisions"}],"predecessor-version":[{"id":6532,"href":"https:\/\/samovar.telecom-sudparis.eu\/index.php\/wp-json\/wp\/v2\/posts\/6531\/revisions\/6532"}],"wp:attachment":[{"href":"https:\/\/samovar.telecom-sudparis.eu\/index.php\/wp-json\/wp\/v2\/media?parent=6531"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/samovar.telecom-sudparis.eu\/index.php\/wp-json\/wp\/v2\/categories?post=6531"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/samovar.telecom-sudparis.eu\/index.php\/wp-json\/wp\/v2\/tags?post=6531"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}