Spring Roo
  1. Spring Roo
  2. ROO-1684

The UTF8 (german and cyrillic) data is not correctly written to the database

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Complete
    • Affects Version/s: 1.1.0.RELEASE
    • Fix Version/s: 1.1.1.RELEASE
    • Component/s: WEB MVC
    • Labels:
      None
    • Environment:
      Windows XP, tomcat

      Description

      The utf8 characters are not correctly written to database.

      a) german special characters (ö,ä,ß,ü)
      b) cyrillic characters (щвды...)

      See
      http://forum.springsource.org/showthread.php?p=327431#post327431
      and
      https://jira.springsource.org/browse/ROO-89?focusedCommentId=60102#action_60102

      I did some tests already, it didnt help:

      1. Set the CharacterEncodingFilter filter in web.xml to first place
      2. Set <property name="hibernate.connection.charSet" value="UTF-8" /> in persistence xml
      3. I updated the DB manually with a correct UTF8 data (copied from hsqldb from same application but in grails) and is showed correctly, so the utf8 data is correcly read\displayed from database.

      I have same application in grails. Same db (hsqldb), db settings, db version.

      Here the sample of data in:

      Grails:
      Entered value "Привет München",
      value in db "INSERT INTO GUETER_EINHEIT VALUES(17,0,'\u041f\u0440\u0438\u0432\u0435\u0442 M\u00fcnchen')"

      Roo:
      Entered value "привет münchen",
      value in the db "INSERT INTO GUETER_EINHEIT VALUES(2,'\u00d0\u009f\u00d1\u0080\u00d0\u00b8\u00d0\u00b2\ u00d0\u00b5\u00d1\u0082 m\u00c3\u00bcnchen',0)"

      Very strange, I see in Roo for one special character two unicode characters and the generated unicode of grails and roo is different

        Issue Links

          Activity

          Hide
          Ro added a comment -

          Hi,
          same problem in my case. None of the workarounds/fixes suggested above was helpful.

          Setting a breakpoint in *Controller.update() method and modifying manually the problematic value to "proper" one just before calling the *.merge() causes the string to be updated to the DB (and then retrieved and updated on the view) correctly.

          Roo 1.1.0.RELEASE
          STS 2.5.0.RELEASE
          MySQL 5.1.51 community

          Berto

          Show
          Ro added a comment - Hi, same problem in my case. None of the workarounds/fixes suggested above was helpful. Setting a breakpoint in *Controller.update() method and modifying manually the problematic value to "proper" one just before calling the *.merge() causes the string to be updated to the DB (and then retrieved and updated on the view) correctly. Roo 1.1.0.RELEASE STS 2.5.0.RELEASE MySQL 5.1.51 community Berto
          Hide
          Ats U. added a comment -

          The description of this issue looks very similar to what I had after updating to Roo 1.1.0.RC1. I solved my problem manually by reordering filter mappings in web.xml as I described when I created ROO-1731 - it might be worth looking at.

          Reordering filters were mentioned here, but note that the order of filter elements is not important - the order of filter-mapping elements are that matter. Maybe that is why some of You didn't get expected results?
          Also I didn't put CharacterEncodingFilter first (just first of those three filter mappings I concidered importaint in the context of ROO-1731) - I had Spring OpenEntityManagerInViewFilter befor those i mentioned in ROO-1731.

          Stefan, if your solution is what I described with ROO-1731, then You could probably close that issue

          Ats U.

          Show
          Ats U. added a comment - The description of this issue looks very similar to what I had after updating to Roo 1.1.0.RC1. I solved my problem manually by reordering filter mappings in web.xml as I described when I created ROO-1731 - it might be worth looking at. Reordering filters were mentioned here, but note that the order of filter elements is not important - the order of filter-mapping elements are that matter. Maybe that is why some of You didn't get expected results? Also I didn't put CharacterEncodingFilter first (just first of those three filter mappings I concidered importaint in the context of ROO-1731 ) - I had Spring OpenEntityManagerInViewFilter befor those i mentioned in ROO-1731 . Stefan, if your solution is what I described with ROO-1731 , then You could probably close that issue Ats U.
          Hide
          Stefan Schmidt added a comment -

          With commit 2f373ffa30c5ceeccd53a9d21e64f6c0006d1d6e the following changes were made:

          Filter mappings are now installed in the following sequence:

          1. CharacterEncodingFilter
          2. HttpMethodFilter
          (3. springSecurityFilterChain)
          4. OpenEntityManagerInViewFilter

          default.jspx contains a pageEncoding page directive: <jsp:directive.page pageEncoding="UTF-8" />

          persistence.xml now contains: <property name="hibernate.connection.charSet" value="UTF-8"/> if Hibernate is selected.

          It would be great if I can get some feedback so we can close this ticket.

          Show
          Stefan Schmidt added a comment - With commit 2f373ffa30c5ceeccd53a9d21e64f6c0006d1d6e the following changes were made: Filter mappings are now installed in the following sequence: 1. CharacterEncodingFilter 2. HttpMethodFilter (3. springSecurityFilterChain) 4. OpenEntityManagerInViewFilter default.jspx contains a pageEncoding page directive: <jsp:directive.page pageEncoding="UTF-8" /> persistence.xml now contains: <property name="hibernate.connection.charSet" value="UTF-8"/> if Hibernate is selected. It would be great if I can get some feedback so we can close this ticket.
          Hide
          Fokko Breden added a comment -

          Stefan, changing filter mapping works. We have moved

          <filter-mapping>
          <filter-name>CharacterEncodingFilter</filter-name>
          <url-pattern>/*</url-pattern>
          </filter-mapping>

          to the first position of all <filter-mapping> entries. Posting UTF-8 is now OK.

          (see also: https://jira.springsource.org/browse/ROO-1731)

          Show
          Fokko Breden added a comment - Stefan, changing filter mapping works. We have moved <filter-mapping> <filter-name>CharacterEncodingFilter</filter-name> <url-pattern>/*</url-pattern> </filter-mapping> to the first position of all <filter-mapping> entries. Posting UTF-8 is now OK. (see also: https://jira.springsource.org/browse/ROO-1731 )
          Hide
          Stefan Schmidt added a comment -

          I'll go ahead and mark this as resolved.

          Show
          Stefan Schmidt added a comment - I'll go ahead and mark this as resolved.

            People

            • Assignee:
              Stefan Schmidt
              Reporter:
              Eugen Eisler
            • Votes:
              5 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: