[Samba] Wrong charset on NTFS volumes (ext4 works fine)

Wed Nov 19 03:52:39 MST 2014

Am 19.11.2014 um 11:31 schrieb Daniel Carrasco Marín:
> El 19/11/14 a las 11:19, Rowland Penny escribió:
>> On 19/11/14 10:11, Daniel Carrasco Marín wrote:
>>> El 18/11/14 a las 21:23, steve escribió:
>>>> On 18/11/14 20:51, Rowland Penny wrote:
>>>>> On 18/11/14 19:39, steve wrote:
>>>>>> On 18/11/14 19:58, Rowland Penny wrote:
>>>>>>> On 18/11/14 18:27, Daniel Carrasco Marín wrote:
>>>>>>>> El 18/11/14 a las 19:05, Jeremy Allison escribió:
>>>>>>>>> On Tue, Nov 18, 2014 at 06:39:55PM +0100, Daniel Carrasco Marín wrote:
>>>>>>>>>> El 18/11/14 a las 18:25, Jeremy Allison escribió:
>>>>>>>>>>> On Tue, Nov 18, 2014 at 12:19:31PM +0100, Daniel Carrasco Marín
>>>>>>>>>>> wrote:
>>>>>>>>>>>> Hi, first of all i'm sorry for my english.
>>>>>>>>>>>>
>>>>>>>>>>>> I've a little problem with samba 3.6.6 and NTFS volume. I've
>>>>>>>>>>>> mounted
>>>>>>>>>>>> an USB drive in NTFS format on my server, and all works fine
>>>>>>>>>>>> (characters are OK, i can create folders with accents or ñ like
>>>>>>>>>>>> "camión", "muñeca"...), but when i try to share that drive through
>>>>>>>>>>>> samba, all special characters look wrong and i can't create any
>>>>>>>>>>>> folder with that characters...
>>>>>>>>>>>> I've a lot of shared folders in ext4 and all works fine, so i think
>>>>>>>>>>>> that the problem is NTFS system.
>>>>>>>>>>>>
>>>>>>>>>>>> I've tried some options like "dos charset", "unix charset" and
>>>>>>>>>>>> "display charset", mounting that drive with "ntfs-3g -o
>>>>>>>>>>>> iocharset=utf8", CP850... but nothing works.
>>>>>>>>>>>>
>>>>>>>>>>>> Someone knows how to fix this?
>>>>>>>>>>> Doesn't seem like a Samba specific problem.
>>>>>>>>>>>
>>>>>>>>>>> What locale is set on your shell when
>>>>>>>>>>> you do an 'ls' and see correct names ?
>>>>>>>>>> The ls command looks fine, all characters are OK when i list files
>>>>>>>>>> or folders, even i can create files/folders with that characters
>>>>>>>>>> without problem. The problem is when i share that files/folders
>>>>>>>>>> through samba.
>>>>>>>>>> In locale i've all in "es_ES.UTF-8".
>>>>>>>>> Then you need to look at debug logs. Samba should
>>>>>>>>> be using the same iconv code that 'ls' is using
>>>>>>>>> internally.
>>>>>>>> Thanks for reply, i've changed the log to debug and I've seen that and
>>>>>>>> filename looks fine:
>>>>>>>> pc007.old:  smbd_dirptr_get_entry mask=[*] found _!-UNIDAD
>>>>>>>> H/__EIA-CAM-Planes-Urbanisticos-Inf-Publica/2014/140728 - PE
>>>>>>>> Tratamiento Afino ETAP - Gri*ñ*on fname=140728 - PE Tratamiento Afino
>>>>>>>> ETAP - Gri*ñ*on (140728 - PE Tratamiento Afino ETAP - Gri*ñ*on)
>>>>>>>>
>>>>>>>> I've tried to create a folder too, but here's where the filename looks
>>>>>>>> wrong too:
>>>>>>>> check_reduced_name: couldn't get realpath for _!-UNIDAD
>>>>>>>> H/__EIA-CAM-Planes-Urbanisticos-Inf-Publica/2014/carpeta sin t*�*tulo
>>>>>>>>
>>>>>>>> Why characters are wrong in share?, because it happen in linux and
>>>>>>>> Windows explorers...
>>>>>>>>
>>>>>>> Hi, are you sure that your 'ntfs-3g' has the option 'iocharset=' , I
>>>>>>> checked the version on my laptop and it doesn't show either with
>>>>>>> 'ntfs-3g --help' or 'man ntfs-3g'
>>>>>>>
>>>>>>> Rowland
>>>>>>>
>>>>>> Hi
>>>>>> Strange. Never tried on an all Spanish box but try with:
>>>>>> -olocale=en_US.UTF8
>>>>>> Si todavía no te va, pues dímelo y lo probaré todo en es_ES.
>>>>>> Salu2,
>>>>>> Steve
>>>>>>
>>>>> hace ntfs-3g --help show iocharset
>>>>>
>>>> Hi
>>>> Yeah. locale. A real pain if your language is not English. Even worse if you need a combination.
>>>> To force the mount, I think these days we need:
>>>> locale=
>>>> Just one other thing that comes to mind from many moons ago when we had a wg was with the Spanish keyboard on windows, e.g. pressing the Ñ without
>>>> the shift lock was not the same as Alt0241 even though both appeared on screen as ñ, only the latter was sent to the printer correctly. We still
>>>> have a dog-eared piece of paper on the notices board:
>>>> á    ALT0225
>>>> é    ALT0233
>>>> í    ALT0237
>>>> ó    ALT0243
>>>> ú    ALT0250
>>>> ñ    ALT0241
>>>> ü    ALT0252
>>>> But don't trust that. I'm sure it's changed several times since then. Oh, and you couldn't have ñ in a filename or login name over samba as I
>>>> remember too.
>>>> HTH and do forgive the nostalgia,
>>>> Steve
>>>>
>>>
>>> Hi again, the problem is near to be fixed.
>>>
>>> The problem was (of course) charset conf in global.
>>>
>>> In global i had:
>>> dos charset = 850
>>> unix charset = ISO8859-1
>>>
>>> And I've changed for:
>>> dos charset = CP850
>>> unix charset = UTF-8
>>> display charset = UTF-8
>>>
>>> and now is working fine in Windows and Linux explorers without problem, but i've seen that others shares have charset problems now...
>>>
>>> Is there any way to set the charset for every share instead global?
>>>
>>> Greetings!!
>>>
>>>
>> Hi, by setting something in the [global] section, it should affect everything, so if you are now have problems with other shares it is unlikely to
>> be a samba problem. Could it be that the files in question, have been saved with the old charset info and now you have corrected the problem, they
>> are being read with the new charset ?
>>
>> Rowland
> Hi,
> 
> Yes, now with that options i can fully work with special characters in that drive without any kind of problem (read, write, create folders,
> rename...), but the others shares have underscores instead special characters... Like a good ext4, i can create folders with special characters in
> that shares without problem, but old folders have wrong characters.
> 
> I need to find a solution for all problems, but i think that is not easy.
> 
> Greetings!!
> 

Samba team member Björn Jacke has written the tool convmv - see
https://www.j3e.de/linux/convmv/man/

which can be used to convert file/dir names from one charset to another.
Many distros allow to install it directly.

Cheers, Günter

-- 