Add An Custom Avatar by a Video or an Image

POST/api/v1/userVideoTwin/startTraining

This API is equivalent to "⚡️Quick Preview" in our website.

Read the tutorial for how to film the best training videos.

Video Mode (Recommended): provide "video_url"

Upload a video of yourself using "video_url". The video will be the base asset for all your AI generated videos. Upload a video of yourself in MP4 or MOV format. It can be either horizontal or vertical, with no size limitations. The video length should be at least 10 seconds. This video will serve as the foundation for all your subsequent AI-generated character videos. Ensure that the person in the video is clear and attractive.
(Optional) If you are unsatisfied with the lip-sync quality of the AI video, please check if your video meets are the requirements (read below). if you uploaded video meets the requirements, you can use "continueTraining" API (requires 1 diamond). Wait for 2 hours, and the avatar AI model will be automatically updated for better lip-sync quality.
Common issues:
- The mime type of your video URL must be set correctly (e.g. video/mp4). We use the mime of the URL header to determine the file type, not the suffix of the URL. If you use an object storage service of a popular cloud service (e.g. S3 of AWS), the mime is usually automatically set.
- No space is allowed in the URL
- Address redirect is not allowed (i.e. 3xx response code of the http request). This is a common issue if someone provides a http link, but later his server redirects the http address to a https address.

Image Mode (Beta): provide "image_url"

Note: The video / image must contain only one face.

Price:
The video startTraining is free. But it taks 1 diamond for 1 "Continue Training" in video mode. If you do not have enough diamonds, check out "Coins and Diamonds" section to learn how to exchange for diamonds.
The image has been updated to version 2, and it now taks 1 diamond for startTraining.

When the training is completed, the API will be automatically uploaded to the user avatar.

Request

Authorization: Bearer Token

Provide your bearer token in the

Authorization

header when making requests to protected resources.

Example

Authorization: Bearer 2f68dbbf-519d-4f01-9636-e2421b68f379

Header Params

x-lang

string

required

the language of you use a2e ai platform. This is not what languange your avatar speaks. If you are unsure about this parameter, use "en-US" by default

Example:

en-US

name

string

required

The name of the video to be uploaded

gender

string

required

The gender of the video to be uploaded. The value is either female or male

video_url

string

optional

The URL of the video of the avatar you want to do AI clone.
Either image_url or video_url should be provided. No space is allowed in the URL.
Video requirements:

Do not use videos with multiple faces appearing.
Ensure the face is neither too large nor too small. The entire face should be within the screen area and not cropped out. It is recommended that the face width occupy between one-tenth and one-third of the overall frame width.
Make sure facial features are not obscured, ensuring the clarity of facial features and contours.
The recommended video resolution is 720P or 1080P, with a maximum resolution not exceeding 4K.
The video duration should be no less than 10 seconds and no more than 5 minutes (10s–5min).
For better lip-sync generation results, it is recommended to use videos of people speaking normally.The audio and lip movements in the video must be synchronized, and background noise or other sounds (except speech) should be avoided. Maintain a moderate speaking speed; speech that is too slow may reduce lip-sync accuracy, while speech that is too fast may cause lip-sync jitter.

image_url

string

optional

The URL of the image of the avatar you want to do AI clone.
Either image_url or video_url should be provided. No space is allowed in the URL.
Image requirements:

The person in the image should be facing forward (angle should allow both ears to be visible).
Ensure there is only one face in the image; images containing multiple faces are not supported.
The face should neither be too large nor too small. Ensure the entire face is within the screen area and not cropped out. It is recommended that the face width occupy between one-tenth and one-third of the overall image width.
Make sure facial features are not obscured, ensuring clear visibility of the facial features and contours.
The image size should not exceed 10MB, and the dimensions should not exceed 4000 pixels in either width or height.

video_backgroud_image

string

optional

The background image of the video. Please checkout our web UI to see how the background image is used.
If you set this value, you do not need to set "video_backgroud_color".

video_backgroud_color

string

optional

The rbg string of the background color of the video. For example, if your avatar video is filmed with a green background, you can set a color value close the green. Then you will be able to do background matting in the following video synthesis stage.
You can film your model in green, white, blue or any pure color background. You must set this color value close to the actual color of the background in your video.
If you set this value, you do not need to set "video_backgroud_image".

ideal_short_edge

integer

optional

The short edge pixels of the avatar, only activated under the image mode. default is 640

skipPreview

boolean

optional

Whether to skip preview and directly perform "continue training".
If set to true, it is equivalent to "Continue Training 💠" in our website.

{
    "name":"dubbing demo",
    "gender":"female",
    // "image_url": "https://d1tzkvq5ukphug.cloudfront.net/adam2eve/stable/video_twin/63076d83-d345-4caa-be8a-19fc7c9338c8.png",
    "video_url":"http://XXXXX/cache/videoplayback%20%2816%29.mp4",
    "video_backgroud_color":"rgb(61,165,82)"
}

Request samples

Responses

startTraining(200)

HTTP Code: 200

Content Type : JSONapplication/json

code

integer

required

data

object

required

name

string

required

gender

string

required

image_url

string

required

video_url

string

required

current_status

string

required

close_mouth_path

string

required

task_status

string

required

video_backgroud_image

string

required

video_backgroud_color

string

required

image_result_url

string

required

sent_time

null

required

preview_result_url

string

required

version

integer

required

isSilent

boolean

required

hasVideoClone

boolean

required

_id

string

required

a.k.a. user_video_twin_id. The ID of the video twin record to query /api/v1/anchor/character_list

createdAt

string

required

updatedAt

string

required

{
  "code": 0,
  "data": {
    "name": "test",
    "gender": "female",
    "image_url": "",
    "video_url": "https://d1tzkvq5ukphug.cloudfront.net/adam2eve/stable/video_twin/2112ce30-ae9a-450f-abf4-6fbf8cab546e.mp4",
    "current_status": "initialized",
    "wl_model": "V3",
    "close_mouth_path": "",
    "task_id": "",
    "task_status": "",
    "video_backgroud_image": "",
    "video_backgroud_color": "rgb(61,165,82)",
    "image_result_url": "",
    "sent_time": null,
    "preview_result_url": "",
    "version": 1,
    "priority": 1,
    "isSilent": false,
    "hasVoiceClone": false,
    "hasVideoClone": false,
    "deleted": false,
    "_id": "66cc39e1b0e0680cadd2ca6e",
    "user_id": "637498294abbb165572d5dbe",
    "createdAt": "2024-08-26T08:16:33.845Z",
    "updatedAt": "2024-08-26T08:16:33.845Z",
    "__v": 0
  }
}

Last modified: 2 days ago