Q-learning cell selection for femtocell networks: Single- and multi-user case

Chaima Dhahri, Tomoaki Ohtsuki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

16 Citations (Scopus)

Abstract

In this paper, we focus on user-centered handover decision making in open-access non-stationary femtocell networks. Traditionally, such handover mechanism is usually based on a measured channel/cell quality metric such as the channel capacity (between the user and the target cell). However, the throughput experienced by the user is time-varying because of the channel condition, i.e. owing to the propagation effects or receiver location. In this context, user decision can depend not only on the current state of the network, but also on the future possible states (horizon). To this end, we need to implement a learning algorithm that can predict, based on the past experience, the best performing cell in the future. We present in this paper a reinforcement learning (RL) framework as a generic solution for the cell selection problem in a non-stationary femtocell network that selects, without prior knowledge about the environment, a target cell by exploring past cells behavior and predicting their potential future state based on Q-learning algorithm. Our algorithm aims at balancing the number of handovers and the user capacity taking into account the dynamic change of the environment. Simulation results demonstrate that our solution offers an opportunistic-like capacity performance with less number of handovers.

Original languageEnglish
Title of host publicationGLOBECOM - IEEE Global Telecommunications Conference
Pages4975-4980
Number of pages6
DOIs
Publication statusPublished - 2012
Event2012 IEEE Global Communications Conference, GLOBECOM 2012 - Anaheim, CA, United States
Duration: 2012 Dec 32012 Dec 7

Other

Other2012 IEEE Global Communications Conference, GLOBECOM 2012
CountryUnited States
CityAnaheim, CA
Period12/12/312/12/7

Fingerprint

Femtocell
Learning algorithms
Channel capacity
Reinforcement learning
Decision making
Throughput

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this

Dhahri, C., & Ohtsuki, T. (2012). Q-learning cell selection for femtocell networks: Single- and multi-user case. In GLOBECOM - IEEE Global Telecommunications Conference (pp. 4975-4980). [6503908] https://doi.org/10.1109/GLOCOM.2012.6503908

Q-learning cell selection for femtocell networks : Single- and multi-user case. / Dhahri, Chaima; Ohtsuki, Tomoaki.

GLOBECOM - IEEE Global Telecommunications Conference. 2012. p. 4975-4980 6503908.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Dhahri, C & Ohtsuki, T 2012, Q-learning cell selection for femtocell networks: Single- and multi-user case. in GLOBECOM - IEEE Global Telecommunications Conference., 6503908, pp. 4975-4980, 2012 IEEE Global Communications Conference, GLOBECOM 2012, Anaheim, CA, United States, 12/12/3. https://doi.org/10.1109/GLOCOM.2012.6503908
Dhahri C, Ohtsuki T. Q-learning cell selection for femtocell networks: Single- and multi-user case. In GLOBECOM - IEEE Global Telecommunications Conference. 2012. p. 4975-4980. 6503908 https://doi.org/10.1109/GLOCOM.2012.6503908
Dhahri, Chaima ; Ohtsuki, Tomoaki. / Q-learning cell selection for femtocell networks : Single- and multi-user case. GLOBECOM - IEEE Global Telecommunications Conference. 2012. pp. 4975-4980
@inproceedings{922dbade638541eba63f7aab9664e2c7,
title = "Q-learning cell selection for femtocell networks: Single- and multi-user case",
abstract = "In this paper, we focus on user-centered handover decision making in open-access non-stationary femtocell networks. Traditionally, such handover mechanism is usually based on a measured channel/cell quality metric such as the channel capacity (between the user and the target cell). However, the throughput experienced by the user is time-varying because of the channel condition, i.e. owing to the propagation effects or receiver location. In this context, user decision can depend not only on the current state of the network, but also on the future possible states (horizon). To this end, we need to implement a learning algorithm that can predict, based on the past experience, the best performing cell in the future. We present in this paper a reinforcement learning (RL) framework as a generic solution for the cell selection problem in a non-stationary femtocell network that selects, without prior knowledge about the environment, a target cell by exploring past cells behavior and predicting their potential future state based on Q-learning algorithm. Our algorithm aims at balancing the number of handovers and the user capacity taking into account the dynamic change of the environment. Simulation results demonstrate that our solution offers an opportunistic-like capacity performance with less number of handovers.",
author = "Chaima Dhahri and Tomoaki Ohtsuki",
year = "2012",
doi = "10.1109/GLOCOM.2012.6503908",
language = "English",
isbn = "9781467309219",
pages = "4975--4980",
booktitle = "GLOBECOM - IEEE Global Telecommunications Conference",

}

TY - GEN

T1 - Q-learning cell selection for femtocell networks

T2 - Single- and multi-user case

AU - Dhahri, Chaima

AU - Ohtsuki, Tomoaki

PY - 2012

Y1 - 2012

N2 - In this paper, we focus on user-centered handover decision making in open-access non-stationary femtocell networks. Traditionally, such handover mechanism is usually based on a measured channel/cell quality metric such as the channel capacity (between the user and the target cell). However, the throughput experienced by the user is time-varying because of the channel condition, i.e. owing to the propagation effects or receiver location. In this context, user decision can depend not only on the current state of the network, but also on the future possible states (horizon). To this end, we need to implement a learning algorithm that can predict, based on the past experience, the best performing cell in the future. We present in this paper a reinforcement learning (RL) framework as a generic solution for the cell selection problem in a non-stationary femtocell network that selects, without prior knowledge about the environment, a target cell by exploring past cells behavior and predicting their potential future state based on Q-learning algorithm. Our algorithm aims at balancing the number of handovers and the user capacity taking into account the dynamic change of the environment. Simulation results demonstrate that our solution offers an opportunistic-like capacity performance with less number of handovers.

AB - In this paper, we focus on user-centered handover decision making in open-access non-stationary femtocell networks. Traditionally, such handover mechanism is usually based on a measured channel/cell quality metric such as the channel capacity (between the user and the target cell). However, the throughput experienced by the user is time-varying because of the channel condition, i.e. owing to the propagation effects or receiver location. In this context, user decision can depend not only on the current state of the network, but also on the future possible states (horizon). To this end, we need to implement a learning algorithm that can predict, based on the past experience, the best performing cell in the future. We present in this paper a reinforcement learning (RL) framework as a generic solution for the cell selection problem in a non-stationary femtocell network that selects, without prior knowledge about the environment, a target cell by exploring past cells behavior and predicting their potential future state based on Q-learning algorithm. Our algorithm aims at balancing the number of handovers and the user capacity taking into account the dynamic change of the environment. Simulation results demonstrate that our solution offers an opportunistic-like capacity performance with less number of handovers.

UR - http://www.scopus.com/inward/record.url?scp=84877672085&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84877672085&partnerID=8YFLogxK

U2 - 10.1109/GLOCOM.2012.6503908

DO - 10.1109/GLOCOM.2012.6503908

M3 - Conference contribution

AN - SCOPUS:84877672085

SN - 9781467309219

SP - 4975

EP - 4980

BT - GLOBECOM - IEEE Global Telecommunications Conference

ER -