Yes you can! These can be managed under Data warehouse -> COGS data by clicking the "Manage Historical COGS" button to the right of a product or variant.
You can then manage costs by date range
*Please ensure that you do not include $ sign when you input the product cost. This will create an error. Only input numeric values and one decimal (0-9)