公开(公告)号：US20240396956A1

公开(公告)日：2024-11-28

申请号：US18791533

申请日：2024-08-01

Applicant: Agora Lab, Inc.

Inventor： Sheng Zhong ,  Tao Duan ,  Jiajie Jian

IPC: H04L65/80 ,  H04L1/16 ,  H04L43/0829

Abstract: A network model for determining the performance of a real-time communication application, such as a video codec and transmission of the coded data, is implemented as computer software. The network model factors in burst data packet loss in determining a performance measure. The network model uses a Hidden Markov Model on a set of data packets of a unit of data to determine the performance measure in the form a probability that all the data packets within the set are received. The measure is used to fine tune the system settings. The network model causes the real-time communication application to adjust parameters of the application for improved transmission of data packets. The adjustment of a parameter can be an increase or a decrease.

公开(公告)号：US12101370B2

公开(公告)日：2024-09-24

申请号：US18195965

申请日：2023-05-11

Applicant: Agora Lab, Inc.

Inventor： Sheng Zhong ,  Jiajie Jian ,  Tao Duan

IPC: H04L65/80 ,  H04L1/16 ,  H04L43/0829

CPC classification number: H04L65/80 ,  H04L1/16 ,  H04L43/0829

Abstract: A system and method for determining the performance of a real-time communication application, such as a video codec and transmission of the coded data. The method factors in burst data packet loss in determining a performance measure. The method uses a Hidden Markov Model on a set of data packets of a unit of data to determine the performance measure in the form a probability that all the data packets within the set are received. The measure is used to fine tune the system settings. The method causes the real-time communication application to adjust parameters of the application for improved transmission of data packets. The adjustment of a parameter can be an increase or a decrease.

公开(公告)号：US12001956B2

公开(公告)日：2024-06-04

申请号：US18319109

申请日：2023-05-17

Applicant: Agora Lab, Inc.

Inventor： Sheng Zhong

IPC: G06N3/084 ,  G06F18/21 ,  G06N20/00 ,  G06T3/4053 ,  G06T5/00 ,  G06V10/776 ,  G06V10/82

CPC classification number: G06N3/084 ,  G06F18/217 ,  G06N20/00 ,  G06T3/4053 ,  G06T5/001 ,  G06V10/776 ,  G06V10/82 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: A method for image generation based on a Generative Adversarial Network (GAN) including a generator, a discriminator, and an encoder, wherein outputs of the generator are mapped, by the encoder, to a latent space adaptable to manipulate at least one characteristics of images generated by the GAN, the method including generating, by the encoder, a first encoding E(Y) of a target image Y and a second encoding E(G(Z)) of a generated image G(Z) corresponding to the target image Y, wherein the first and second encodings E(Y) and E(G(Z)) map Y and G(Z) to the latent space having a lower dimension than dimensionality of Y and G(Z), wherein the encoder is trained to minimize the differences between the first and second encodings E(Y) and E(G(Z)), and the generator is trained by using the first and second encodings E(Y) and E(G(Z)) as part of a loss function.

公开(公告)号：US11694085B2

公开(公告)日：2023-07-04

申请号：US17324831

申请日：2021-05-19

Applicant: Agora Lab, Inc.

Inventor： Sheng Zhong

IPC: G06K9/62 ,  G06N3/084 ,  G06T3/40 ,  G06N20/00 ,  G06T5/00 ,  G06F18/21 ,  G06V10/776 ,  G06V10/82

CPC classification number: G06N3/084 ,  G06F18/217 ,  G06N20/00 ,  G06T3/4053 ,  G06T5/001 ,  G06V10/776 ,  G06V10/82 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: A method of training a generator G of a Generative Adversarial Network (GAN) includes receiving, by an encoder E, a target data Y; receiving, by the encoder E, an output G(Z) of the generator G, where the generator G generates the output G(Z) in response to receiving a random sample Z and where a discriminator D of the GAN is trained to distinguish which of the G(Z) and the target data Y; training the encoder E to minimize a difference between a first latent space representation E(G(Z)) of the output G(Z) and a second latent space representation E(Y) of the target data Y, where the output G(Z) and the target data Y are input to the encoder E; and using the first latent space representation E(G(Z)) and the second latent space representation E(Y) to constrain the training of the generator G.

公开(公告)号：US20210211768A1

公开(公告)日：2021-07-08

申请号：US17212552

申请日：2021-03-25

Applicant: Agora Lab, Inc.

Inventor： Sheng Zhong ,  Yue Feng

IPC: H04N21/442 ,  G06F3/041 ,  G06F3/01 ,  G06K9/00 ,  H04N21/234 ,  H04N21/44 ,  H04N21/84 ,  G10L15/18 ,  G10L25/63 ,  G10L25/57 ,  G10L15/22

Abstract: A method, an apparatus, and a system for video communications include: transmitting, from a first apparatus using a network, a first video stream of a first user to a second apparatus of a second user, wherein the first user is in video communication with the second user; receiving, from the second apparatus using the network, a second video stream of the second user; determining, by a processor, a reaction of the second user to an area of interest in the first video stream using the second video stream; and updating, in response to the reaction of the second user to the area of interest in the first video stream, a parameter for encoding the area of interest in the first video stream at the first apparatus.

公开(公告)号：US10779012B2

公开(公告)日：2020-09-15

申请号：US16209939

申请日：2018-12-04

Applicant: Agora Lab, Inc.

Inventor： Sheng Zhong ,  Wei Dai

IPC: H04N19/895 ,  H04N19/625 ,  H04N19/176 ,  H04N19/159 ,  H04N19/61

Abstract: A method, an apparatus, and a system for error concealment of video communications include determining, by a processor in an encoding process, a current block of a current slice of a current picture of a video sequence for error concealment, determining recovery data of the current block, wherein the recovery data comprises at least one of a coefficient of a discrete transform performed to the current block and similarity data associated with a first block similar to the current block, and the first block is in a first slice different from the current slice, and transmitting the current block and the recovery data using a network, wherein the current block is transmitted as in-band data and the recovery data is transmitted as out-of-band data.

公开(公告)号：US20200177926A1

公开(公告)日：2020-06-04

申请号：US16209939

申请日：2018-12-04

Applicant: Agora Lab, Inc.

Inventor： Sheng Zhong ,  Wei Dai

IPC: H04N19/895 ,  H04N19/625 ,  H04N19/61 ,  H04N19/159 ,  H04N19/176

Abstract: A method, an apparatus, and a system for error concealment of video communications include determining, by a processor in an encoding process, a current block of a current slice of a current picture of a video sequence for error concealment, determining recovery data of the current block, wherein the recovery data comprises at least one of a coefficient of a discrete transform performed to the current block and similarity data associated with a first block similar to the current block, and the first block is in a first slice different from the current slice, and transmitting the current block and the recovery data using a network, wherein the current block is transmitted as in-band data and the recovery data is transmitted as out-of-band data.

公开(公告)号：US12101458B2

公开(公告)日：2024-09-24

申请号：US18093788

申请日：2023-01-05

Applicant: Agora Lab, Inc.

Inventor： Wei Dai ,  Sheng Zhong ,  Tao Duan ,  Zesen Zhuang ,  Xiajun Gu

IPC: H04L65/80 ,  H04N17/00 ,  H04N19/124 ,  H04N19/89

CPC classification number: H04N17/00 ,  H04N19/124 ,  H04N19/89

Abstract: A system and method for determining the performance of a video codec for real-time communication applications. A set of network conditions is used to simulate certain typical real-world network conditions. A coded video stream is transmitted from the transmitting end to the receiving end under these conditions. The end-to-end latency and received video fluency are measured along with a set of existing video quality measures by a video codec performance evaluation system including a network model and a video codec quality analyzer. The decodable frame ratio, latency and video fluency are used as the performance metrics for real-time communication video quality evaluation. The video codec performance evaluation system does not send any video data over a network when it determines the real-time communication quality of the video codec.

公开(公告)号：US20230421626A1

公开(公告)日：2023-12-28

申请号：US18195965

申请日：2023-05-11

Applicant: Agora Lab, Inc.

Inventor： Sheng Zhong ,  Jiajie Jian ,  Tao Duan

IPC: H04L65/80 ,  H04L43/0829 ,  H04L1/16

CPC classification number: H04L65/80 ,  H04L1/16 ,  H04L43/0829

Abstract: A system and method for determining the performance of a real-time communication application, such as a video codec and transmission of the coded data. The method factors in burst data packet loss in determining a performance measure. The method uses a Hidden Markov Model on a set of data packets of a unit of data to determine the performance measure in the form a probability that all the data packets within the set are received. The measure is used to fine tune the system settings. The method causes the real-time communication application to adjust parameters of the application for improved transmission of data packets. The adjustment of a parameter can be an increase or a decrease.

公开(公告)号：US10567757B2

公开(公告)日：2020-02-18

申请号：US15993868

申请日：2018-05-31

Applicant: Agora Lab, Inc.

Inventor： Sheng Zhong ,  Bin Zhao

IPC: H04N19/105 ,  H04N19/164 ,  H04N19/174 ,  H04N19/65

Abstract: A method for encoding a video stream, including encoding, by a processor, a first slice of a first picture of the video stream; receiving, from a decoder, a feedback message indicative of having received the first slice of the first picture; and in response to receiving the feedback message indicative of having received the first slice of the first picture, updating a dynamic reference picture (DRR) with the first slice of the first picture, wherein the first slice of the first picture replaces the first slice of a second picture in the dynamic reference picture, and the first slice of the first picture and the first slice of the second picture are co-located.