US 11,704,559 B2
Learning to search user experience designs based on structural similarity
John Collomosse, San Jose, CA (US)
Assigned to Adobe Inc., San Jose, CA (US)
Filed by Adobe Inc., San Jose, CA (US)
Filed on Jun. 17, 2020, as Appl. No. 16/904,460.
Prior Publication US 2021/0397942 A1, Dec. 23, 2021
Int. Cl. G06N 3/08 (2023.01); G06N 3/04 (2023.01); G06F 18/2135 (2023.01); G06N 3/045 (2023.01); G06F 8/38 (2018.01); G06F 16/583 (2019.01)
CPC G06N 3/08 (2013.01) [G06F 8/38 (2013.01); G06F 16/583 (2019.01); G06F 18/21355 (2023.01); G06N 3/04 (2013.01); G06N 3/045 (2023.01)] 16 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
generating a graph representation of a layout of a graphical user interface (GUI), the layout including a plurality of control components, each control component including a component class, geometric features, and relationship features to at least one other control component, the graph representation including a plurality of nodes corresponding to the plurality of control components and at least one edge connecting the plurality of nodes;
processing the plurality of nodes of the graph representation by a first one or more layers of a graph convolutional network (GCN) to generate a plurality of node embeddings;
processing the at least one edge of the graph representation by a second one or more layers of the GCN to generate a plurality of relationship embeddings;
generating a search embedding for the representation of the layout based on the plurality of node embeddings and the plurality of relationship embeddings; and
querying a repository of layouts in embedding space using the search embedding to obtain a plurality of layouts based on similarity to the layout of the GUI in the embedding space.