In this paper, a packet construction mechanism using an ID-based factorial number structure (ID-based FNS) is proposed for a secure conference call system in an ad hoc network. We adopt the ID-based concept for a conference key generation in which each participant with one’s own public identity uses a pre-chosen secret-key to generate the exchange conference key for the conference call. Furthermore, due to the mobility of nodes in an ad hoc network environment, it is likely that some nodes quit or join during the operation of this conference. To solve these problems, we employ an information exchange table for each participant node in order to manage a secure conference call in an ad hoc network system. In addition, our scheme employs only one cryptographic algorithm of the secret-key system. The extraction of the secure sub-packets associated with each intended receiver is protected on the basis of the Diffie–Hellman key exchange protocol with the trusted public-key exchange. Therefore, the subsequent message recovery is successfully accomplished. This provides a new and feasible solution for a secure conference call in an ad hoc network system.