| CPC G06V 20/47 (2022.01) [G06V 10/761 (2022.01); G06V 20/41 (2022.01)] | 8 Claims |
|
1. An attention-based video summarization method comprising:
extracting frame-level visual features from an input video;
computing an attention weight and representing an importance score as a frame tracking probability for selecting a key frame by using the attention weight;
obtaining a temporal consistency reward function and a representativeness reward function so as to select the key frame, based on a visual similarity distance and temporal distance between key frames, and training an attention-based video summarization network to predict an importance score for selecting a key frame of a video summary by using the temporal consistency reward function and the representativeness reward function;
creating a video summary by selecting a corresponding key frame based on the predicted importance score, evaluating the quality of the created video summary, and performing policy gradient learning for the attention-based video summarization network;
calculating regularization and reconstruction loss for controlling the probability to select a key frame by using the importance score of the selected key frame; and
creating a video summary based on the calculated regularization and reconstruction loss.
|