Write to user defined file with PowerShell

I'm working on a general solution for PowerShell job execution on SQL Server Agent. This generates a lot of small tasks, which mostly will be described here with a solution.

Right now it's about logging, and for this I'm thinking in several directions.
This task is for a solution using files, that are defined in the executing script.

The script writes to the same file in each execution, but the file will only contain the last execution:
$FileName = $MyInvocation.MyCommand.Name.split(".")[0] + ".Log"
Write-Host $FileName
"Hello to file..." > $FileName
"Another hello to file" >> $FileName

The execution generates this content (Get-Content Write2File.Log):
Hello to file...
Another hello to file
Another execution gives the same content - two lines, not four.